{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1031abed-175d-5ec5-8396-88c6d66d2d92","task_key":"default--validation--24447","task_revision_id":"1","upstream_id":"24447","short_description":"HellaSwag validation 24447","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to collect water [title] Know the sources. [step] There are many ways to collect water, and some are more accessible or sanitary than others. Consider the following places from which people often draw their water : [substeps] Rainfall: you can collect and store rainwater.\",\"endings\":\"['Rainwater is important for health and nutrition as well as dehydration and wellbeing. Irrigation: the irrigation system which collects water.', 'You can then deposit it on your garage door, or in the store closet beneath a car hood. Other sources you can collect water from are lakes, rivers, river beds, pvc pipes, air-conditioning systems, cable-tight garbage cans, woodshops and utility vehicles.', 'It is usually pure enough for drinking, cooking, and bathing. Beware, however, of the chemical contaminants in \\\" acid rain.', \\\"Rainwater is produced for many uses, including disinfecting and disposing of fish and reptiles. If you see water collection issues, you'll learn a lot about water consumption potential.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}