{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3447f319-33cd-5a61-9f76-4735284c774d","task_key":"default--validation--41727","task_revision_id":"1","upstream_id":"41727","short_description":"HellaSwag validation 41727","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to find wild horses [title] Understand what you're looking for. [step] Most \\\" wild \\\" horses are actually feral horses, or domestic breeds that are no longer domesticated. Real wild horses are very rare.\",\"endings\":\"[\\\"Some wild horses, like larger prairie horses or other breeds that are domesticated, may not have raised or redomesticated before domesticated. This is because they tend to actively treat wild horses while they're housed with them.\\\", 'The przewalski horse, found in mongolia, is one of the only real wild horses left in the world. Most free-roaming horses are actually feral, not wild.', \\\"They are not found in abundance country-dwellers or bandits. Wild animal panginates won't be wild being groups of humans, cows, goats, goat, sheep, pigs, mixed breeds and countless others.\\\", '[title] Understand horses \\\" unwilling \\\" to be left unsupervised. [step] Tamed horses are not allowed out into the wilderness, because of the dangers and know that they are not banned for being near animals that are too tame.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}