{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"10783acd-e8a5-50aa-8b63-739b62e2bfb9","task_key":"default--validation--40783","task_revision_id":"1","upstream_id":"40783","short_description":"HellaSwag validation 40783","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to melt chocolate chips [title] Fill a saucepan with 2 inches (5.08 centimeters) of water. [step] You can use an actual double boiler, or improvise your own with a saucepan and a heat-safe bowl. Do not place the second saucepan or bowl on top just yet, however.\",\"endings\":\"['[title] Melt one cup (240 milliliters) of milk in a saucepan over medium heat. [step] You can use multiple a cup (240 milliliters), or you can use the entire pot.', '[title] Place the chocolate chips into the saucepan and simmer them for 5 minutes. [step] This will create a process called \\\" processing, \\\" which means that the chocolate chips will break apart if they are not melted quickly enough.', '[substeps] If you are using a bowl, make sure that it fits comfortably over the saucepan and that there are no gaps where heat can escape. Double boilers are great if you need the chocolate to stay melted for a long time (ie: when you are making chocolate dipped strawberries).', '[substeps] The heat of the water will help melt the chocolate chips. Make sure that the temperature of the water is roughly nine to 12 degrees fahrenheit (4 to 5 degrees celsius).']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}