{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"553f6ba0-d233-5f68-b98b-57b5f6a6434b","task_key":"default--validation--27309","task_revision_id":"1","upstream_id":"27309","short_description":"HellaSwag validation 27309","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to drive safely in heavy traffic [title] Remove all distractions. [step] In heavy traffic conditions, the road will be crowded with cars, the flow of traffic irregular, and people will start getting impatient, leading to them trying to merge where they likely shouldn't. The last thing you need is a distraction preventing you from paying attention to these things.\",\"endings\":\"['[title] Place safety items around your car at all times. [step] This will provide yourself a lot of protection.', \\\"There's no need to doze into your seat, or drive without headphones. You can even turn your cell phone off when driving to keep yourself on task.\\\", 'Limit your distractions by : [substeps] Turning off your cell phone, or putting it on silent mode. Switching off your music, or turning the volume down.', '[substeps] Put your phone away and consider leaving your phone on silent so as not to pick it up on the lane. You can also try turning on your alarm, to keep it in silent mode.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}