{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"09eef129-d150-5abd-a05c-aa2059ac7ff3","task_key":"default--validation--47839","task_revision_id":"1","upstream_id":"47839","short_description":"HellaSwag validation 47839","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to cook radishes [title] Preheat the oven to 450 ° f (232 ° c) and grease a baking sheet. [step] Get out a rimmed baking sheet and drizzle it with a little olive oil. Use a pastry brush to evenly coat the bottom of the sheet with the oil to prevent the radishes from sticking.\",\"endings\":\"['[title] Peel and core 2 radishes. [step] Use a vegetable peeler to peel and core 1 radish.', '[substeps] Alternatively, you can grease the sheet with olive oil instead. [title] Wash and wash the radishes.', '[substeps] You could also use canola, vegetable, or coconut oil. [title] Rinse and cut the radishes.', \\\"[substeps] If you don't have the time or patience to prep your radishes right now, you can just steam them. This can make cooking the radishes easier and save time on the baking sheet.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}