{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ee112db4-35d0-504f-9426-09937a4d6a0a","task_key":"default--validation--45850","task_revision_id":"1","upstream_id":"45850","short_description":"HellaSwag validation 45850","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to react if a power line falls on your car [title] Stay inside the car if possible. [step] Unless your car is on fire, it's very important to stay inside your car once a power line falls. When a power line falls on your car, it often charges the metal parts of the car with electrical current, meaning that if you attempt to exit as you normally do, you will be electrocuted.\",\"endings\":\"[\\\"[substeps] After the line falls, look around the inside of your car and out the windows. If you don't see flames or smoke, you aren't in any immediate danger.\\\", 'If you drop the power line, it will blow up. If you cannot exit as you normally would, call the police.', 'The higher the voltage, the more powerful the car will be, making it much more difficult to get away after it falls. Walk away from the car slowly to go back to the car.', \\\"This occurs during the fall, and you don't want to get electrocuted at all. [title] Determine if you have lightning or thunder on your car.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}