{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f553a490-043d-5e15-93ca-af591486259f","task_key":"default--validation--36787","task_revision_id":"1","upstream_id":"36787","short_description":"HellaSwag validation 36787","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to be safe during a cruise ship fire [title] Participate fully in any muster drill. [step] This is an exercise designed for passengers to practice moving safely and quickly during an emergency to prevent injury and save lives. Passengers learn how to use life vests and about escape routes on their ships.\",\"endings\":\"['Participate as much as possible in any muster drills. [title] Stay calm and mind your own safety.', 'Muster drills are required by international law and for good reason. While they may seem like a nuisance during a vacation they are done for your own safety and well-being.', '[substeps] Some obstacles, such as fire, are difficult to get out of, but not impossible to overcome. Passengers are trained in how to navigate them and to recognize when and how an obstacle may close.', '[title] Know that basic tasks are as follows : [substeps] Wear good glasses. A wide brimmed hat is usually good.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}