{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9c1286ac-029f-5562-a003-619c58ef90e3","task_key":"default--validation--13599","task_revision_id":"1","upstream_id":"13599","short_description":"HellaSwag validation 13599","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to arrange refrigerator shelves [title] Put your fruit in the low humidity drawer. [step] Fruit keeps best when it isn't exposed to too much moisture. Most refrigerators have a special drawer that has the lower humidity than the other shelves and drawers.\",\"endings\":\"[\\\"So it's best to place your fruit in this drawer, as you're less likely to hear insects from outside when temperatures rise. [substeps] The drawer can also have an escape ventilation screen that prevents insect pollination.\\\", 'Sometimes it\\\\'s labeled \\\" low humidity, \\\" and other times it\\\\'s labeled \\\" crisper. \\\" this is where you should store your fruit, from apples to bananas to grapes.', 'If you are heating or storing fruit in the cooler, place your fruit in this special drawer and lock the doors and windows. [substeps] Extra storage space is key for most refrigerators, but some refrigerators also have multiple compartments so that you can keep fruit in one compartment.', 'Be sure to place your fruit in this drawer. [title] Choose a deep-sealed freezer container.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}