{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3d33e2df-e754-5c4e-98d2-48e0e1f9b7ed","task_key":"default--validation--43404","task_revision_id":"1","upstream_id":"43404","short_description":"HellaSwag validation 43404","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to heal from a collapsed lung [title] Get to the emergency room. [step] Go to the doctor or emergency room immediately if you experience sudden chest pain, or any other symptoms of a collapsed lung like difficulty breathing, nasal flaring, chest tightness, and easy fatigue. [substeps] If there was any blunt trauma to your chest, a doctor should be seen if shortness of breath and chest pain occurs, or if you cough up any blood.\",\"endings\":\"['A collapsed lung can be caused by a variety of factors. Most often, it is the result of trauma to the chest or ribcage.', '[title] Remove heavy objects that weigh down your chest. [step] Examine the contents of your chest and take measurements based on the size and shape of objects you moved prior to the collapsed lung.', '[title] Check for breathing difficulties for those who have collapsed. [step] Breathe difficulties for patients with flat airways, heart issues, or heart problems will often affect their breathing.', 'If you experience any of these symptoms, call emergency services right away. Coughing will help prevent you from triggering a choking spell again.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}