{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ac8e22f9-dd4e-5abc-b0fc-4c23e52d5452","task_key":"default--validation--23886","task_revision_id":"1","upstream_id":"23886","short_description":"HellaSwag validation 23886","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to chlorine wash a pool [title] Drain your pool. [step] Before you can chlorine wash your pool, you will need to drain it. Drain your pool using the waste line of your pool pump and motor system.\",\"endings\":\"['Pour your trash into your waste bin, and close your drain lid. Then, pour water through your drain hose into the garbage.', \\\"Go to the gas pump and turn it on to drain the pool. Then add your pool's waste hose into the drain.\\\", 'Alternately, hose your pool and turn off the faucets. Let the pool drain for 1-2 minutes if you wish to pool wash it more often.', '[substeps] Check your water discharge regulations before draining your pool so you can dispose of the water you pump out in a responsible manner. Many municipalities prohibit pumping water into street sewers.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}