{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1ec923c8-3469-5291-b110-358663ef635d","task_key":"default--validation--23495","task_revision_id":"1","upstream_id":"23495","short_description":"HellaSwag validation 23495","config":"default","split":"validation","body":"{\"activity_label\":\"Home and Garden\",\"ctx\":\"[header] How to melt chocolate in the microwave [title] Chop the chocolate into small pieces with a serrated knife. [step] If you try to melt a whole chocolate bar, the outside will melt before the center, and the chocolate will be more likely to scorch. By chopping up the chocolate into small pieces, it will melt more evenly.\",\"endings\":\"['[substeps] You can also use a cookie cutter, or instead of a cookie cutter, you could use a butter knife. If you used butter, be sure to scrape away the inside or the outside of the chocolate.', '[substeps] You can use any type of chocolate, but gold or tropical chocolate is the most common. [title] Place the chocolate on a microwaveable sheet.', \\\"So don't leave the chocolate out if you're planning on serving it or if you wish to bake it. [substeps] The smaller the pieces, the easier this will be.\\\", \\\"[substeps] You can skip this step if you're using chocolate chips. [title] Place the chocolate into a microwave-safe bowl.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}