{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f7449d0a-2ebb-5ac9-97e2-393f5f103434","task_key":"default--validation--5033","task_revision_id":"1","upstream_id":"5033","short_description":"HellaSwag validation 5033","config":"default","split":"validation","body":"{\"activity_label\":\"Sports and Fitness\",\"ctx\":\"[header] How to frolic [title] Find an area fit for frolicking. [step] This can be an open space such as a park, your backyard, or a rich friend's living room that has more space. Anywhere is good, as long as it doesn't constrict you or limit your mobility.\",\"endings\":\"[\\\"[title] Don't train in cold conditions. [step] The cold cold air and cold temps affect your circulation and also decrease strength in the muscles.\\\", \\\"[title] Don't be ashamed! [step] Frolicking is a natural thing. If you're hesitant to frolic in a public place, go ahead and find somewhere cozy and private.\\\", 'Wherever you are, try to choose an area that may cover a lot of space. [substeps] For instance, a pond or creek with other bodies of water is a great area.', '[substeps] Keep the area where you frolic soft and shady. Cover the whole area with old papers or paper towels.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}