{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"589383df-9dde-5906-a76c-5bbc3cdd3812","task_key":"default--validation--25578","task_revision_id":"1","upstream_id":"25578","short_description":"HellaSwag validation 25578","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to deal with anorexic friends or family [title] Have them see a doctor about diagnosis and treatment. [step] Untreated anorexia nervosa can become a serious health issue, potentially causing kidney problems, heart problems, and even death. If they exhibit the following symptoms, ask them to see their primary care physician as soon as possible.\",\"endings\":\"['If they are present when treatment begins, the doctor may check them. [substeps] Swelling of the lymph nodes.', 'Your doctor may suggest medication such as morphine or ibuprofen if any of these do not help their symptoms. [substeps] They may recommend ssris if symptoms persist for more than six months.', \\\"[substeps] Have this person visit your doctors with an accurate diagnosis. If they've had this episode before, they may be alarmed to see you, or they may feel drawn to someone with this disorder.\\\", 'If you are their parent, take them to see the doctor. [substeps] Eating very little and avoiding fatty foods.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}