{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"290dc2ca-d79a-5392-93b5-96bb2a820aaf","task_key":"default--validation--13629","task_revision_id":"1","upstream_id":"13629","short_description":"HellaSwag validation 13629","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to make popcorn balls [title] Bring the light corn syrup, butter, water, confectioners' sugar, and marshmallows to a boil over medium heat. [step] Pour the light corn syrup into a saucepan, then stir in the butter, water, confectioners' sugar, and marshmallows. Heat the mixture over medium heat until it starts to boil.\",\"endings\":\"[', stirring often to prevent scorching or burning. [title] Put the popcorn into a large bowl, then stir the mixture into it with a rubber spatula.', '[substeps] Pour enough water into the saucepan to just cover the marshmallows. You can add more or less, depending on your desire.', 'Be sure to cover the pan tightly to prevent the popcorn from sticking to it. [title] Simmer the popcorn for 45 seconds.', \\\"[substeps] If you don't have a saucepan, you can just use a ceramic plate. Keep in mind that it will take a lot longer for the popcorn balls to melt.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}