{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0f3f0961-4f62-54cc-bbec-df0cd167a9cc","task_key":"default--validation--16023","task_revision_id":"1","upstream_id":"16023","short_description":"HellaSwag validation 16023","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to strip turf [title] Cut the turf. [step] Use a lawnmower to cut the turf as short as possible. [title] Cover the turf.\",\"endings\":\"['[step] Cut short side to side as well as deeper, consistent cuts so as to create best results and fade around the turf. Keep running your lawnmower as it will suck up dirt and debris.', '[step] Apply beer or rain on the turf if it is wet or wet. In the winter, layer the turf around it to keep the ground moist.', '[substeps] Lay corrugated cardboard, or between 6 and 8 layers of newspaper, over the turf to be stripped. Ensure that there is no colored ink on the newspaper or cardboard.', '[step] Cut off 100% of the turf and let it sit next to the wind away from the road. This will loosen dirt and debris, allowing for clean-up.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}