{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e618bda2-7789-58fd-a7c9-2a0ba64547a1","task_key":"default--validation--18637","task_revision_id":"1","upstream_id":"18637","short_description":"HellaSwag validation 18637","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to melt hershey kisses [title] Unwrap the kisses and place them in a heat-safe bowl. [step] Hershey kisses come wrapped individually in thin foil. The standard milk chocolate versions, which are the best for melting, are wrapped in plain silver foil.\",\"endings\":\"['Use store-bought, homemade kisses if you wish. [substeps] If you like homemade ones, wrap them in foil, though you may wish to crush the chocolate.', '[title] Fill a microwave-safe bowl with 4 ounces (200 ml) of water and 1 ounce (15 ml) of sugar. [step] You can microwave the dipped kisses for just a few seconds for extra sweetness.', 'Warm the kisses gently in the microwave at least 30 seconds before opening and unfolding them or they may melt before melting. When you open and reunwrap your kisses, the chocolate will liquefy and melt.', 'Unwrap each one and place them into a heat-safe bowl (like glass). If you aren\\\\'t sure, check your bowl for a \\\" heat-safe \\\" label.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}