{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d9caa25d-f2e6-5799-a97f-600d9ebd9619","task_key":"default--validation--18912","task_revision_id":"1","upstream_id":"18912","short_description":"HellaSwag validation 18912","config":"default","split":"validation","body":"{\"activity_label\":\"Cars & Other Vehicles\",\"ctx\":\"[header] How to remove brake dust from aluminum wheels [title] Park on a safe, flat surface with good drainage. [substeps] Put your vehicle in park with the emergency brake up. Don't park on a steep slant like a hill.\",\"endings\":\"[\\\"Although unlikely, rolling is always a concern with any sort of vehicular maintenance. Since you'll be using soap or commercial wheel cleaners for this method, you won't want to park somewhere where your rinse water will run into a storm drain.\\\", '[title] Remove your u-turn and chain the wheel to the ground. [title] Attach the safety equipment to the tow chain.', 'The brake can cause a collision with the back of the vehicle. [title] Remove as much of the dust from the wheels as possible using a flashlight or a glove container.', 'You want to make sure the power and air flow are good. Use rainwater on the wheels to improve drainage.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}