{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"762695e1-b48e-57e5-a5de-f00542b4c450","task_key":"default--validation--23737","task_revision_id":"1","upstream_id":"23737","short_description":"HellaSwag validation 23737","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to make gingerbread biscotti [title] Preheat the oven and prepare the baking sheets. [step] Turn on the oven to 350 degrees f (176 c). Spread a light layer of shortening or cooking spray onto a few baking sheets.\",\"endings\":\"['Spread the melted butter (or margarine) on the baking sheets. [substeps] You can use a butter box, but remember to coat it evenly with cooking spray to ensure that it reaches the bottom of the baking sheets easily.', 'Set them aside while you make the biscotti. [substeps] You could also line the sheets with a silicone mat or parchment paper.', 'Spread around the edges to avoid having them sticking while baking. If the baking sheet is plastic, pour some of the flour into the bottom of the dish.', \\\"Sprinkle flour on one of them, then wipe down the surfaces with a clean rag. [substeps] You'll need to use something like a pastry brush to spread flour over the surfaces so they stay evenly moist.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}