{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"499921c2-7684-5036-8266-bf67f96e7a73","task_key":"default--validation--4770","task_revision_id":"1","upstream_id":"4770","short_description":"HellaSwag validation 4770","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to clean dr brown bottles [title] Dissemble the bottle. [step] To effectively clean any bottle, it's important to take it apart. However, dr. brown's bottles have more parts than traditional baby bottles, so you must be sure to fully disassemble it.\",\"endings\":\"['Make a show of all the parts inside, including the bottom and the back sides of the bottle. [substeps] You can also wear gloves and an apron to get access to the inside.', 'Then, use a screwdriver to loosen the screw on the screw on the inside. Gently pull the bottle apart so that the screw and screw head meet.', 'Use a sharp knife to carefully remove the bottle. [substeps] Remove as much air as possible from the bottle, even if its in the refrigerator.', \\\"Separate the cap, nipple collar, nipple, vent insert, vent reservoir, travel disc, and any insert components from the bottle so you can clean each part. [substeps] If you're not sure how to take apart your particular dr.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}