{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"91fd8d5b-9f3f-55c0-82ab-2b407e577b4d","task_key":"default--validation--21520","task_revision_id":"1","upstream_id":"21520","short_description":"HellaSwag validation 21520","config":"default","split":"validation","body":"{\"activity_label\":\"Cars & Other Vehicles\",\"ctx\":\"[header] How to repair electric car windows [title] Locate and open the fuse box. [step] It should be located inside the car as a part of, or near, the dash. [title] Check your owner's manual to locate that operates the windows.\",\"endings\":\"[\\\"[step] The manual may have been written by an experienced mechanic in previous years or have a reliable list that lists all of the steps your car is going to take to repair electric windows. [substeps] On your driver's door panel, sometimes under the armrest, you may see a row of buttons or knobs for different air vents.\\\", '[title] Remove the clamps at the bottom of the window. [step] Gently push them down under the window opening as though removing heavy objects.', '[step] It is not uncommon that the only thing wrong with electrical components in a vehicle is a blown fuse. If this is the case with your windows you will need to replace the window fuse.', \\\"[step] If the model has an owner's manual or modified unit that allows for wiring to run through the box, flip it over and pull the wire that's holding the fuse box in place. The wiring will not be visible while the window is open.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}