{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4790108a-2fe3-5268-8451-0d68c9d1828e","task_key":"default--validation--25101","task_revision_id":"1","upstream_id":"25101","short_description":"HellaSwag validation 25101","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to stop a dog from climbing up on things [title] Immediately react when your dog climbs onto something. [step] When your dog climbs up onto the counter, you need to immediately react. Dogs learn by testing their boundaries.\",\"endings\":\"[\\\"Here are some responses to observe : [substeps] Bark click or bark yip shove, bite, or lick your dog's mouth squeal [title] Hold a wet dog on a table instead of a pad. [step] This is a safer approach than chasing a skittish, small dog, but it's effective.\\\", 'This means that \\\" prying \\\" or jumping back from something will encourage it to climb closer to you. Ensure that the object is safe before immediately rushing to hop on.', \\\"If they have been up on these spaces for longer than a couple of seconds, they might not understand why you are telling them to get down. Rather than associating your command with their location, they might just begin to think that you're a mean person-someone to be feared.\\\", \\\"They'll learn important characteristics that, if raised incorrectly, can seem impossible to obtain while trying to climb. If your dog climbs onto something unstable, it's likely to fall and what you do get is loose wear and tear.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}