{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"90139c84-e755-5930-b5fd-7a65109fd954","task_key":"default--validation--35875","task_revision_id":"1","upstream_id":"35875","short_description":"HellaSwag validation 35875","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to raise a healthy bunny [title] Give your rabbit unlimited quantities of hay. [step] Hay forms the foundation of a rabbit's diet. Not only is this important for nutrients, but fiber as well.\",\"endings\":\"['Since rabbits clean themselves, they often swallow a lot of hair. Without adequate fiber, these hairballs can block the gi tract and can be fatal.', \\\"Don't go too many hours without hay-or at least solid hay, at least once a day. Hay will provide insulation from the weather, as well as aid the rabbit's digestive system.\\\", 'When rabbits are young, they will need the hay every other day. Purchase or make hay in large, square, or cone baskets that you can easily roll a blanket into.', 'Your rabbit will not eat hay if it has food poisoning. To avoid this, provide your rabbit ¼ cup of hay daily.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}