{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ef13510b-b45b-57d1-a2e5-660d070939bb","task_key":"default--validation--6654","task_revision_id":"1","upstream_id":"6654","short_description":"HellaSwag validation 6654","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to make cookies without flour [title] Use nuts or oats instead of flour. [step] Flour acts as a binder to keep your cookies together. To replace flour as a binding agent, replace flour in your recipe with chopped nuts (like pecans, macadamia nuts, or walnuts) or rolled oats.\",\"endings\":\"['[substeps] To save money, skip this portion of the baking process entirely. [title] Cook in heated oil instead of butter.', \\\"You can also substitute nuts for butter when using butter to help transfer flour from the food to your cookie dough. [substeps] If you won't be using any real oats in your dough mixture, use them instead of flour.\\\", 'These will bind your other ingredients together and help the cookies hold their shape as they bake. [substeps] Try a no-bake chocolate oatmeal cookie or traditional flourless almond cookies.', '[substeps] Use bittersweet, unsalted dry flour instead of your favorite dry for a more cost-effective substitute. [title] Use oven-safe sugar substitute instead of white flour.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}