{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"745c5546-f20f-558e-9c76-28551bf61c5d","task_key":"default--validation--4754","task_revision_id":"1","upstream_id":"4754","short_description":"HellaSwag validation 4754","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to treat diabetes related hypoglycemia [title] Get a glucometer. [step] A glucometer is a small, battery-powered medical device used to check the glucose level in your blood. [substeps] If you don't have a glucometer at home, you can have your blood glucose level checked at a hospital.\",\"endings\":\"[\\\"You will then be transported to the doctor's office where a glucometer can measure glucose levels and can send these results to the doctor. It's best to order glucometers in bulk, as these are not as pricey anymore.\\\", 'Most hospitals do not provide glucometers. You may need to ask your doctor or pharmacist.', 'Each glucometer comes with test strips. Extra test strips can be purchased at any drugstore.', \\\"The equipment involved is almost completely automated, so you don't have to talk to anyone about it. Glucometers cost between $100 and $$ 50 usd each.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}