{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"42c22445-099a-5df8-841a-937e20dc15f0","task_key":"default--validation--16600","task_revision_id":"1","upstream_id":"16600","short_description":"HellaSwag validation 16600","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to can tomatoes [title] Choose your tomatoes. [step] Any variety of tomatoes can be used, but make sure the fruit is not too ripe. Overripe tomatoes are bad for canning due to their high acid content.\",\"endings\":\"['[substeps] If you can find fresh tomatoes, cut them into quarters or slices. Make sure your cutting board is clear and that you can picture the fruits.', 'Gently press on the tomatoes to make sure they are still firm. Look over your tomatoes to make sure they do not have any bruises.', '[title] Use chinese cabbage, lettuce or kale. [step] Ask your butcher to repot or feed your vegetables any cabbage you choose.', 'That said, you can always have loose tomatoes cook over the stove, or simmer over the stove without them if that is the case. [substeps] Dried fruits are also good tomatoes to use for canning.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}