{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9f964e24-f37b-5f1b-a1e2-a079e3bcf429","task_key":"default--validation--10414","task_revision_id":"1","upstream_id":"10414","short_description":"HellaSwag validation 10414","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to have a tidy linen closet [title] Keep like with like. [step] It makes good common sense to keep the same items together in the linen closet. For example : [substeps] Put all the bed linen together.\",\"endings\":\"['Put all the linens that are in it together, as well as whites, blankets, pillowcases, stuffed animals, handkerchiefs, etc. Put the clothes into one pile and underwear/clothing in another pile.', 'Keep the linen bedroom blanket on the bed. [title] Declutter rooms with matching linens.', 'Some people prefer to put the sheet set together (fitted sheet, flat sheet, pillow cases) and keep it intact. Others prefer to separate into fitted sheet piles, flat sheet piles, pillowcase piles.', 'Throw some big throw pillows on the bed. Add a rug or two in the corner for consistency.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}