{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"831c89a4-4e0b-5985-818a-55ae132a17a4","task_key":"default--validation--37240","task_revision_id":"1","upstream_id":"37240","short_description":"HellaSwag validation 37240","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to set up a vivarium for a bearded dragon [title] Buy a vivarium. [step] You can buy your vivarium from a pet store. You can also check online sellers and auction sites, like ebay.\",\"endings\":\"['[title] Use an aquarium to house the dragon. [step] Bearded dragons are breeding fish, making the vivarium a special breeding ground for one.', 'Check that the vivarium is large enough to keep the bearded dragon resting and you can properly rule out child or adult dragons. [title] Make a table or tote for the bearded dragon to rest on.', 'There are glass, pvc, melamine, abs, and vision cages to choose from. [substeps] A traditional vivarium has wooden sides with a sliding glass frontage.', '[title] Buy another cage from a trustworthy seller. [step] You can also make your own vivarium by getting a different cage from a third-party seller.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}