{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3c489ca1-b362-5a70-905e-845b97ec1a92","task_key":"default--validation--12134","task_revision_id":"1","upstream_id":"12134","short_description":"HellaSwag validation 12134","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to throw up without being embarrassed [title] Recognize the symptoms that precede vomiting. [step] Many people notice nausea before they are sick. Nausea is the feeling that you might be sick.\",\"endings\":\"['The nausea may rise when you drop liquids or do things that can make urine difficult to pass through your intestines. You might feel nauseous if the symptoms are related to your illness, like vomiting or diarrhea.', 'It is usually less subtle or unpleasant than the feeling when you are nauseated or terrified. Other symptoms include : [substeps] Abdominal pain bad-smelling urine pale skin nausea numbness [title] Stop vomiting or get medical treatment to prevent nausea while you are sick.', 'Most people also experience over-activity in their parotid glands just prior to regurgitating, which causes salivation. By producing extra saliva, the body is helping to reduce the acidity of the vomit.', \\\"It spreads throughout body, including the digestive system. Nausea for instance, feels like a sharp sensation that doesn't go away after three hours of sweating or will otherwise cause or worsen nausea, diarrhea, and vomiting.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}