{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"631bab01-240a-5d73-a7c7-93f46b4cdd5a","task_key":"default--validation--23732","task_revision_id":"1","upstream_id":"23732","short_description":"HellaSwag validation 23732","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to care for superworms [title] Choose a container. [step] A plastic container can work well for superworms. You can also pick up an aquarium at a local pet store.\",\"endings\":\"[\\\"A dark-colored container with drainage holes will also work. Since superworms rarely burrow or crawl underground, it's best to find a tank that is specifically made for superworms.\\\", 'The plastic container will prevent the thousands of superworms per bag. A larger container will also prevent the hundreds of eggs and larvae that hatch from superworms from escaping the container and forming the worms.', 'The container should be large enough that its top will be at least three inches away from the substrate. [substeps] You can easily use a plastic container you have lying around the house for superworms.', 'Your container should be big enough to comfortably hold hundreds of superworms, which will be frozen overnight. While the container can be wide, you should be able to fill it up to your standard size.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}