{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4a12d1dc-be2d-57e7-b17c-4188132c2b11","task_key":"default--validation--29958--variant-5d5186a95b5c","task_revision_id":"1","upstream_id":"29958--variant-5d5186a95b5c","short_description":"HellaSwag validation 29958","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to gloss wood [title] Clean visible dirtiness with a damp cloth. [step] For a little extra cleaning power, add a dot of mild soap, like dish detergent. Wipe the wood thoroughly until it is clean.\",\"endings\":\"[\\\"Follow the damp cloth with a dry one to remove any remaining moisture. When the wood is clean and dry, you're ready to sand.\\\", 'Have your carpenters create their own designs with this type of cleaning product. [title] Wipe away dust from the wood with a soft cloth.', '[title] Wipe your wood with a rag dampened with water. [step] Wipe the wet rag across the entire surface of the wood in a 50 degree motion.', '[title] Coat the wood in soapy water. [step] After cleaning, wet a rag with cool to lukewarm water.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}