{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"19e3f482-0bd0-5f8a-a097-350fd19c3125","task_key":"default--validation--32747","task_revision_id":"1","upstream_id":"32747","short_description":"HellaSwag validation 32747","config":"default","split":"validation","body":"{\"activity_label\":\"Cars & Other Vehicles\",\"ctx\":\"[header] How to remove car battery terminals [title] Open the hood of the car and use the support bar to prop it open. [title] Locate the battery within the engine compartment. [step] If you're unsure about which part is the battery, then check your owner's manual.\",\"endings\":\"['[title] Slide the body of the battery off of the alternator. [step] Then disconnect the alternator with a pair of pliers.', 'Most vehicles have bottom drain hoses, but hoses in your engine should also be labelled with this information. To make it easier, just tug the bottom of the hose hose out of the compartment.', 'Some vehicles have their battery in the trunk, under or behind an access panel. [title] Check the positive post to be sure that the cover is on.', '[title] Review the tab on the left sidewall of the car and see if there is a loose screw or another resistant locking mechanism. [step] Sometimes the battery actually lies in the contact area.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}