{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ff906f66-9bcd-5add-b6b6-c7c744dbda3b","task_key":"default--validation--43619","task_revision_id":"1","upstream_id":"43619","short_description":"HellaSwag validation 43619","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to choose wedding shower games [title] Play wedding shower bingo. [step] Create and print grids of 16 squares on card stock or sturdy paper, and then choose 24 words to randomly write on the cards. These words will answer questions you will ask about the bride or groom.\",\"endings\":\"['[substeps] Type \\\" wedding shower bingo \\\" in the \\\" search wedding ideas \\\" field at the top-right of the page, then click the wikihow \\\" marriage bingo \\\" button to get started. [title] Get the bride or groom together in front of the bathroom mirror.', \\\"For example, each area of the wedding should include lights and balloons. [substeps] Hold each division of the wedding like you're holding two cards in the middle of a strip of card stock.\\\", 'Whoever gets four in a row wins! You can ask questions like : [substeps] Where did the couple meet? What did the groom study in college? Where will the couple go on their honeymoon? [title] Try wedding movie charades. [step] This game works like classic charades, but guests will mime classic wedding movies.', 'These words must include : [substeps] The bride is married married and the groom is a united states citizen. The bride is married and the groom is a lawful citizen of the united states.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}