{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bfe09707-e3f3-5c8e-b213-82f4cba206bc","task_key":"default--validation--21668","task_revision_id":"1","upstream_id":"21668","short_description":"HellaSwag validation 21668","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to calm a vicious rabbit [title] Show the rabbit it's hurting you if it bites. [step] Make a sudden yelping sound or squeal when and if your rabbit bites or nips you. This will act as a kind of signal that you're in pain, and your rabbit will associate that with biting you.\",\"endings\":\"['Be sure to discourage biting due to its obedient nature. [substeps] You can also make it less likely that your rabbit will bite you by rubbing it against something, such as a tree trunk, a post (or other object), or a cat.', \\\"[substeps] Place the muzzle over your rabbit's mouth, blocking its airways, and apply firm but gentle pressure. Continue holding the muzzle in place until your rabbit calms down.\\\", \\\"[substeps] Tiny nips can just be your rabbit's way of telling you to go away, or that you're bugging it. They're not trying to hurt you, just trying to let you know they don't want to be touched or handled.\\\", '[title] Scold the rabbit if it bites or nips you. [step] If the rabbit bites you, you should immediately bring it to a veterinarian for medical attention.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}