{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"01e308ae-481a-5bec-9202-1eed9284bfdb","task_key":"default--validation--13402","task_revision_id":"1","upstream_id":"13402","short_description":"HellaSwag validation 13402","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to diagnose chronic inflammatory demyelinating polyneuropathy (cidp ) [title] Experience primary physical symptoms. [step] Cidp is closely related to guillain barré syndrome (gbs), and many of the symptoms are similar. The first signs that a person may have cidp include \\\" motor deficiencies, \\\" like physical weakness and pain.\",\"endings\":\"['These symptoms usually progress slowly over a period of two months. It is important to note that these symptoms could indicate a wide-variety of treatable conditions, and do not alone indicate cidp.', 'The symptoms usually do not last for more than a few weeks, but can last several months. Call your doctor or consult a health care provider if you think that you may have cidp.', 'They also may experience fatigue or joint tension. They may also have obstructions, such as constipation or inability to work.', 'Symptoms can include : [substeps] Shaking: it hurts when you touch the top of your head. This pain might be hard to pinpoint, but it will be very strong when discontinuing activities.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}