{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f6ee8626-2ec6-58a0-8109-10ccb2061f22","task_key":"default--validation--168","task_revision_id":"1","upstream_id":"168","short_description":"HellaSwag validation 168","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to choose a water dish for your rabbit [title] Avoid plastic bowls. [step] Your rabbit's bowl should be made of ceramic or metal. Plastic bowls are easily scratched.\",\"endings\":\"['[title] Choose a soft surface for your rabbit to hide in. [step] Rabbits like to hide on soft, fluffy surfaces.', 'The bottom of a plastic or metal bowl can be damaged, so make sure that your rabbit is not using them. Plastic bowls are safe for humans and are easy to clean out and replace.', 'Bacteria can then build up in the crevices and contaminate the water. Your rabbit may also pick up a plastic bowl and treat it as a toy.', \\\"Instead, use a stainless steel bowl to reduce scratching on your rabbit's paws. [title] Provide a water bowl for your bunny.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}