{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f9db069d-58c9-5e73-83e0-f8b3754c1ed2","task_key":"default--validation--27385","task_revision_id":"1","upstream_id":"27385","short_description":"HellaSwag validation 27385","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to care for a leopard gecko [title] Buy a 10-20 gallon (37.9-75.7 l) tank with a screen lid. [step] Purchase a glass or plastic container from a pet store to house your leopard gecko, including a securely fastened screen lid. A secure lid is important, especially if you have a cat.\",\"endings\":\"['A lid will enable your leopard gecko to vocalise safely, and provide protection against harmful items. [substeps] Both reptiles and geckos need a lid that is wide enough that they can crawl and jump up and down in.', '[substeps] Leopard geckos should be kept in cages with which a single door opens. Bedding should be kept inside the tank, as leopard geckos do not need a lot of room to move and breathe.', \\\"You may find these sold as aquariums, vivariums, or terrariums. If you already have a home for your gecko, skim through this section to make sure it meets your pet's requirements.\\\", 'Your gecko will not want to be disturbed during the day, and it will also need a bright light to get around in the dark. Large geckos may also require bigger screen screens.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}