{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"90fcdb58-24cd-50ab-9873-5fc0f77adaf5","task_key":"default--validation--8912","task_revision_id":"1","upstream_id":"8912","short_description":"HellaSwag validation 8912","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to bottle feed babies [title] Gather your equipment. [step] Before you are able to bottle feed you will need to purchase bottles, nipples, a bottle brush, and a sterilizer. A sterilizer is optional if you have a dishwasher that has a steam setting.\",\"endings\":\"['If your dishwasher does not have a steam setting, then you will need to purchase a pair of sterilized cotton balls and a clean syringe. Also remember that you will be using this brush to administer weaning.', 'There are many types of bottles and nipples you can choose from. Some babies take to only one type of nipple and may you go through many nipples trying to figure out which one they like and latch on too.', 'If not, your access to the sterilizer should be limited, or restricted. [substeps] Sterilizers can be purchased for \\\" non-food \\\" equipment, but also for household implements like a toilet.', \\\"[title] Purchase a blanket or cloth diaper. [step] If you are concerned about babies' bodies getting wet, you can sterilize a blanket or cloth diaper for a younger baby.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}