{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7f646228-027e-57d9-9dbb-6386a6b67de7","task_key":"default--validation--5727","task_revision_id":"1","upstream_id":"5727","short_description":"HellaSwag validation 5727","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to dry a dog [title] Allow the dog to air dry naturally. [step] This is the easiest and most enjoyable way for the dog to dry. After getting wet, the natural reaction of many dogs is to shake from side to side with great vigor to release the water.\",\"endings\":\"['Dry the dog by waving a towel near the dog and holding it in front of him as you shake. Dry dogs naturally shake after an accident, while dirty dogs are especially prone to shake from side to side.', \\\"This helps air to flow through the dog's body and speeds the drying process. [substeps] Letting your dog dry upside down with its chest out can help with this.\\\", 'In less than 4 seconds of shaking, dogs can shake up to 70% of the water from their fur. [title] Stand back and allow the dog to shake the water from his or her fur.', 'To encourage this, your dog has to naturally dry themselves faster because you are taking a short nap before allowing him to dry itself. [substeps] Wait until after a shower or bath to air dry your dog.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}