{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4d7c8237-8c41-56dc-9eb5-c1e04f5a51d7","task_key":"default--validation--36346","task_revision_id":"1","upstream_id":"36346","short_description":"HellaSwag validation 36346","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to prepare for your new guinea pig [title] Purchase or build a cage. [step] Guinea pigs can be housed in three different types of cages; c & c, hutches or runs. You can purchase these from warehouse/pet stores or build your own.\",\"endings\":\"['[substeps] Set the size of the cage you buy according to the kind of environment you will be housing it in. For example, a cage that is 70 by 90 centimeter (183.6 by 183.7) is a good size for guinea pigs because they will be able to run around inside and hold each other.', 'It is important that the cages that you build are sturdy and roomy. Wooden cages that can be suspended from stakes are the perfect option.', \\\"The cage must be at least 4 square feet for each guinea pig housed in it (the more the better) or at least 8 square feet for two. [substeps] Don't fall into the trap of buying inappropriate sized cages from pet shops that do not hold enough space for guinea pigs.\\\", 'Each type of cage differs in size from the other. [substeps] There is a typical 10 square foot cage made from wire mesh (uti).']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}