{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2cb3d9cf-b107-5010-9de2-600422ff6194","task_key":"default--validation--9473","task_revision_id":"1","upstream_id":"9473","short_description":"HellaSwag validation 9473","config":"default","split":"validation","body":"{\"activity_label\":\"Cars & Other Vehicles\",\"ctx\":\"[header] How to drive defensively [title] Avoid distractions. [step] Defensive driving means focusing on one thing: driving. Make sure that you're not getting distracted by conversations, the radio, your phone, or anything else.\",\"endings\":\"[\\\"Concentrating on one thing, every few seconds, will ensure that you don't stay on the phone while you're driving. [substeps] This also applies when you're making side-of-road decisions or looking out the window or watching television.\\\", 'The focus should not be on what others are talking about, but on your driving. [substeps] Do not let your phone get distracted by the radio.', \\\"To be as alert as possible, put your phone and other distractions away and turn your radio down. If you're distracted when talking, don't participate in conversations until you reach your destination.\\\", 'Instead, listen to birds chirping, go for your bike, or even break out your favorite board game. [substeps] Instead of following the instructions on the dashboard, just listen carefully to the instructions as the driver applies them.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}