{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7e8a19d0-843d-590a-a563-6546fa185efa","task_key":"default--validation--42945","task_revision_id":"1","upstream_id":"42945","short_description":"HellaSwag validation 42945","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to sew a scrunchie [title] Find a hair tie. [step] Try to get one of those fabric coated hair ties instead of a rubber band. They are much sturdier, and will last must longer.\",\"endings\":\"['[title] Get the scrunchie as well. [step] Rubber band scrunchies work for both larger and smaller circles of hair on the head.', \\\"You don't have to buy an elastic, as these are made out of natural fibers and should not feel stiff. Therefore, don't use one, but do choose one instead of a rubber band.\\\", '[title] Place your hair either on top of your neck or around your shoulders. [step] This is the most basic scrunchie-like position.', '[title] Cut a rectangle out of fabric that is 4 inches (10.16 centimeters) wide and 18 inches (45.72 centimeters). [step] For a plain scrunchie, use some cotton or jersey knit fabric.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}