{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"345dc890-d1af-5959-88ba-4fe0c134d5a7","task_key":"default--validation--20009--variant-b9ab6f26fba5","task_revision_id":"1","upstream_id":"20009--variant-b9ab6f26fba5","short_description":"HellaSwag validation 20009","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to make chocolate cake with ganache [title] Preheat the oven to 350ºf/180ºc and gather all of the ingredients. [title] In a mixing bowl, whisk together the flour sugar, butter, and baking powder. [step] After the dry ingredients are incorporated, whisk the eggs in one at a time.\",\"endings\":\"['[title] When everything is mixed well, add the chocolate syrup. [step] Whisk until the batter is smooth.', 'With a spatula, separate the eggs. Once the eggs are mixed, add the cold milk and whisk again.', \\\"[title] Pour in the milk and cream mixture (you'll need more if you're using a large bowl) and beat to combine. [title] Sift the milk, cream, and vanilla into the bowl.\\\", '[title] Add the sugared strawberries to the egg mixture and combine. [step] In another mixing bowl, blend all of the eggs 1 at a time.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}