{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9e81626a-0efa-5001-a79b-910911f8cead","task_key":"default--validation--42461","task_revision_id":"1","upstream_id":"42461","short_description":"HellaSwag validation 42461","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to diagnose multiple myeloma [title] Notice if have chronic nausea, fatigue, and a lack of appetite. [step] You may feel mentally or physically weak due to a lack of appetite and experience significant weight loss because you are not eating regularly. [title] Check if you have constant bone pain and frequent illnesses.\",\"endings\":\"['[step] Your bones may feel sore, inflamed, or painful. You may also be susceptible to contracting infections and illnesses due to the multiple myeloma.', \\\"[step] You can also have other conditions causing you to have symptoms over time. [title] Note if you experience irregularities in your liver's enzymes.\\\", '[step] According to 272 doctors, myeloma is diagnosed in 75% of cases. Others would have never noticed these symptoms or been concerned.', '[step] One symptom of myeloma may be the appearance of odd bones. Your doctor may be able to get some bones from certain areas of your body, such as corners of your nose or corners of the mouth.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}