{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"862205af-b801-5b54-8f77-cedc0a9a13c4","task_key":"default--validation--17323","task_revision_id":"1","upstream_id":"17323","short_description":"HellaSwag validation 17323","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to manage multiple sclerosis as you age [title] Take your medication as directed. [step] As you age, it is important to follow your treatment plan for your ms. As soon as you are diagnosed, you will likely be put on drugs to help manage your ms.\",\"endings\":\"['Taking your medication as directed can help keep symptoms, flare ups, and the progression of the disease at a minimum. [substeps] If you need reminders, place notes around your house.', 'Do not take high doses of bad drugs like methadone, lycopene, or over-the-counter medication. During in the weeks leading up to your ms visit, consult with your doctor to make sure your ms has not been ignored.', 'The idea is to ensure that you are careful and take the right dosage of medication to help promote proper early clinical function. [substeps] The proper dosage for ms affects 10% of people with mcat syndrome.', 'You will need to take as many as your doctor advises you. Most ms medication is divided into two doses: lsd, xanax and tricyclic acid.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}