{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1a8ac64e-f558-535c-9c2c-abd185825b87","task_key":"default--validation--32428","task_revision_id":"1","upstream_id":"32428","short_description":"HellaSwag validation 32428","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to hold your syrian hamster [title] Let your hamster get used to his environment. [step] If you have just brought your syrian hamster home, he is going to need some time to adjust to his new surroundings. Let him explore his cage for a day or two before you start trying to hold him.\",\"endings\":\"['You can place him outside in your yard or in a different area. Your hamster may be very excited at the prospect of eating something that him and his owner did not buy.', '[substeps] You may decide to allow him to sleep in your crib until he warms up. You may then take him on brief walks every day.', 'Then, take him outside to a dog park and take him for walks around the neighborhood. [substeps] Make sure to bring a leash and harness as well, as this will help prevent your syrian hamster from bolting away when let out of his cage.', \\\"During this time, he will mark his territory by washing his face and grooming a lot (' scenting'). [substeps] Do not mistake scenting for nervousness or anxiety.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}