{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"04a6864c-4aca-5b90-b0ab-e2b8b55c025f","task_key":"default--validation--14935","task_revision_id":"1","upstream_id":"14935","short_description":"HellaSwag validation 14935","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to wrap a turban [title] Select your material. [step] It should be about 1.5 m by 0.75 m. A large, lightweight scarf works best.\",\"endings\":\"['You can wrap a scarf around a bottle cap or a wine bottle. Lightweight materials can be found at many craft stores as well as online.', '[title] Determine whether you want to wrap more tightly or loosely. [step] Wrap a turban around itself several times.', 'The bigger the material you use, the larger the turban will be, so choose accordingly. Also, thinner materials will be easier to handle.', 'This is especially important for formal occasions when the dampness or moisture inside your turban will stop them from being able to work fully well. Whatever material the scarf is made of, it will need to be strong enough to prevent it from splitting or shrinking if not wrapped.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}