{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1be0fae2-80c4-56a6-927a-f76746abb363","task_key":"default--validation--43642","task_revision_id":"1","upstream_id":"43642","short_description":"HellaSwag validation 43642","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to restrain a rabbit [title] Approach the rabbit in a slow, calm manner. [step] Rabbits scare easily, so it is important to go slow when you pick up your rabbit. Try not to make any sudden movements or do anything else that might scare the rabbit.\",\"endings\":\"[\\\"[substeps] For example, do not yell, jump up and down, or thrust your hand into the rabbit's cage. Instead, try to speak in a low voice, approach the cage at a slow and steady pace, and then slowly reach into the rabbit's cage.\\\", \\\"[substeps] If you want to restrain the rabbit, slowly approach the rabbit and pet it. You can also use a rope or leash to help you gain control of the rabbit's tail.\\\", 'The rabbit may not react with physical physical action, and may also be nervous about being picked up. When calming a rabbit, try saying soothing things like \\\" good boy, it\\\\'s okay \\\" or \\\" no, henry, it\\\\'s okay, \\\" or putting your hand over its nose and calling it to you.', 'Your steps should do most of the work for you. Do not restrain the rabbit by placing one hand under its collar or by the scruff of its neck (this is generally not difficult to do without supervision).']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}