{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c4f525fb-a550-56ad-a3c0-dd8e6467c0a9","task_key":"default--validation--40917--variant-17ede2498f36","task_revision_id":"1","upstream_id":"40917--variant-17ede2498f36","short_description":"HellaSwag validation 40917","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to make stollen [title] Preheat the oven to 190c , 375f or gas mark 5. [step] ( note that you will not need the oven hot until later in the process, so you can start without it on. ) [title] Warm the milk until you can just dip your little finger in it.\",\"endings\":\"['[step] The vinegar will boost the flavour so it will also taste more indian, and you will be able to enjoy it with breakfast. The milk mixture may likewise be used for frying, bake or steaming, or both if you want.', '[step] Note: it may take some milk to get the lumps out. [title] Spoon it into a small bowl (with two sides facing each other) or try to go more watery, the milk will get into the crevices and soften them up.', '[title] Pour the milk into a glass jug. [step] Add 1 teaspoon of the sugar along with the dried yeast and leave it until it forms a frothy head about 2.5 cm (1 inch).', '[step] This is your visual palette, so make sure you use a variety of colors. For example, if you want to get dramatic colors for your stollen, try getting some milk on your finger first and then adding butter or oil.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}