{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"97de3265-3ef6-5292-a921-23f84ae325d0","task_key":"default--validation--20645","task_revision_id":"1","upstream_id":"20645","short_description":"HellaSwag validation 20645","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to train your cat to fetch a toy mouse [title] Pick the right sized toy mouse. [step] In general, cats like to fetch things that they can easily grab with their paws or put in their mouths. If you do not already have a toy mouse at home, you can purchase one at your local pet store.\",\"endings\":\"['Cats will enjoy playing with smaller toys, like paper snowflakes. [substeps] To put the toy mouse into the right mouth, position your cat in your lap and place one paw on top of the other.', 'Consider the size of your cat as you are deciding which toy mouse to purchase-a kitten would need a smaller toy mouse than an adult cat. [substeps] If possible, pick a toy mouse that does not have plastic eyes.', \\\"These are small rubber balls with bones on them that, depending on the size of your mouse mouse, can be bigger than its normal size. It's important to make sure that the toy mouse is not too small for the mouse mouse.\\\", 'Most cats go for small, lightweight toys that you can put in their mouths. Make sure your toy mouse is thin enough so it is easy to pick up.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}