{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bdfaf32c-02cb-5915-9d7e-09851068e6a5","task_key":"default--validation--45167","task_revision_id":"1","upstream_id":"45167","short_description":"HellaSwag validation 45167","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to heal ulcers fast [title] Make a doctor's appointment to determine the cause of your ulcer. [step] The type of treatment you need will depend upon what is causing your ulcer. If you don't already know the cause, talk to your doctor about diagnostic options.\",\"endings\":\"['They may suggest laboratory testing, endoscopy, and/or x-rays, and will be able to refer you to any necessary specialists. [title] Use antibiotics to treat h pylori infections.', '[substeps] For example, your doctor might suggest congestive heart failure, bleeding out from the ulcer site, or an abscess that would have caused your ulcer. Your doctor may also recommend surgery, especially if the ulcer is very deep and serious.', 'Your doctor can help you formulate a plan and help you determine the best form of treatment. [substeps] If you suffer from gum disease, ulcers will likely require surgery.', \\\"[substeps] It's best to have had you treated 6 times each year by a doctor who has experience treating sids. Your doctor can suggest doctor's visits to treat ulcers in other areas of your body so they can formulate the appropriate treatment options for you.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}