{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"67cf3e1f-51f1-5db7-8177-5b875cd300db","task_key":"default--validation--42824","task_revision_id":"1","upstream_id":"42824","short_description":"HellaSwag validation 42824","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to set up a fish tank (for goldfish ) [title] Consider the size of your fish tank. [step] Goldfish require particularly roomy living arrangements in order to stay healthy. While they are pretty small fish, they need larger tanks than you might expect.\",\"endings\":\"[\\\"[substeps] You can do better than a bowl. Despite the beauty of a goldfish suspended in a sphere of glass, most fishbowls simply don't provide enough room for their occupants.\\\", 'The fish will require more room to move around during the day than they would in an aquarium. [title] Choose a small tank for the goldfish to retreat into.', \\\"Goldfish can only live in one tank, so it's best if you have enough extra fish or larger tank plants to accommodate a wide variety of fish. [substeps] Remember that goldfish are not omnivores.\\\", \\\"For example, if you have a 20 gallon tank and 4 fish, you'll likely need 20 gallons. [substeps] If you have a larger tank, consider a 3 foot (90 cm) tank.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}