{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f5f4c27d-63f0-50dc-bfcc-76f9fe88bec6","task_key":"default--validation--40935","task_revision_id":"1","upstream_id":"40935","short_description":"HellaSwag validation 40935","config":"default","split":"validation","body":"{\"activity_label\":\"Travel\",\"ctx\":\"[header] How to walk safely at night [title] Keep your head up. [step] When walking alone at night, make sure to keep your head up and look forward. This will help you be alert and aware of your surroundings.\",\"endings\":\"[\\\"If you are in a particularly dark area, you won't be able to see through the dark. [substeps] If you have a pair of goggles and such, you can always turn on a flashlight to make sure you can see through the lenses at night.\\\", 'Do not look down or away, and only make brief eye contact with others. [substeps] To avoid looking distracted, do not walk with your cell phone in your hand.', \\\"Studies show that walking alone sometimes causes drowsiness. [substeps] Don't look out for fallen leaves, twigs, or other places to hide.\\\", \\\"Consider wearing headbands to keep your head up and even more alert during your walk. Make sure those band holders are securely in place so that you don't lose them.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}