{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"58c128ac-a101-54da-b57f-dc8fc3a7a955","task_key":"default--validation--1860--variant-e320170fd2c7","task_revision_id":"1","upstream_id":"1860--variant-e320170fd2c7","short_description":"HellaSwag validation 1860","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to make a kruschicki cookie [title] Beat egg yolks, sour cream, white sugar, vanilla, and whiskey in large bowl until smooth. [title] Stir in flour slowly until dough is firm enough to knead. [title] Knead dough on a lightly floured cutting board until smooth.\",\"endings\":\"['[title] In an instant mixer add 3 eggs , 1 teaspoon sugar , 2 teaspoons vanilla, and 1 tablespoon flour. [title] Stir the ingredients in thoroughly until well mixed.', '[title] Roll dough until thick, about 1/3 full. [title] Fold in the top half of the dough and smooth down with an even roll.', '[title] If dough is soft and squishy, knead in more flour until dough is firm and sticky. [title] Wrap dough in wax paper and place in a sealed plastic bag.', '[title] Roll flat dough out on 2 tablespoons of coconut, half or three tablespoons of creamy coconut. [title] Transfer the dough to the bowl with the butter and knead dough until smooth.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}