{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"36640a1c-8756-5a8f-ae3f-965c26a76c2a","task_key":"default--validation--12585--variant-e22377224c79","task_revision_id":"1","upstream_id":"12585--variant-e22377224c79","short_description":"HellaSwag validation 12585","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to make churro apple pie bowls [title] Preheat the oven to 450 ° fahrenheit or 225 ° celsius. [title] Grease a muffin tray by flipping it on its side. [step] Instead of spraying non-stick greasing spray in the cases of the tray, grease it on the cups sides.\",\"endings\":\"['[substeps] The churro bowls are made onto the cups of the muffin tray. To prevent it from sticking, it must be sprayed on that area.', '[title] Add the batter and butter to the muffin tray. [step] Spread the flour over the contents of the tray while cooking.', 'Some of the liners should also be greased along the sides. [title] Add the filling ingredients except the butter into the empty muffin tray.', 'Spread the muffin mixture out from the inner rim of each cup and sprinkle the rim lightly with flour. [substeps] You can use a non-stick greasing spray made from butter or non-stick baking flour to help break down larger, gooey chunks of dough.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}