{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"41c77261-a6ef-5e0e-b480-db47fcb5d32a","task_key":"default--validation--20892--variant-07fcf1862ff0","task_revision_id":"1","upstream_id":"20892--variant-07fcf1862ff0","short_description":"HellaSwag validation 20892","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to plan a play date for your blind or visually impaired child [title] Choose a theme. [step] If you want to plan a sensory play party for your blind or visually impaired child and their friends, a theme can help keep the activities simple and understandable. Your theme should be a common thread running through all the activities in some way.\",\"endings\":\"[\\\"[substeps] Your theme doesn't have to include fancy decorations, or novelty things like airplanes or hot dogs. You can also set up activities around a general theme, like a historical restaurant or a haunted house.\\\", \\\"[substeps] The theme doesn't have to be something fancy or exotic. Focus on an outdoor theme, like a rainbow barn or deer habitat.\\\", 'It might be a landmark or simply something that somehow appeals to the blind or visually impaired child. [substeps] Your theme will probably be something simple like animal themed games, activities that differ from one another, or things that are very sexual.', '[substeps] For example, you might want to have a construction party, and use objects related to building and construction work, such as blocks, hard hats, shovels, and other machines. Think not just of occupations, but also places.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}