{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"236f51b0-8be2-51e8-a60a-6c12aa58e1d2","task_key":"default--validation--46935","task_revision_id":"1","upstream_id":"46935","short_description":"HellaSwag validation 46935","config":"default","split":"validation","body":"{\"activity_label\":\"Finance and Business\",\"ctx\":\"[header] How to get the most from a car accident claim [title] Stay calm. [step] In order to maximize the amount that the insurance company will pay for your claim, it is very important to pay attention at the scene of the car accident. Getting into a car accident is a stressful situation, but try to stay calm.\",\"endings\":\"['[title] Check to see if anyone is injured. [step] If you or the other driver is injured, help as much as you can until paramedics arrive.', 'You want the insurance company to help you to understand why the accident took place, so stay calm and then try to keep driving. [substeps] Let the information come into your mind.', 'Look out for warning signs, such as doors slamming and headlight flashing. [substeps] If you are in the police or ambulance, take the opportunity to talk to the police officer in charge of the accident.', '[substeps] If your friend or family member is hurt, call 911 or have them call a cab or an ambulance. If the site of the accident is flood-prone, help the person or people attending the accident stay inside of the vehicle until help arrives.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}