{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"057ebe72-9c8c-579d-83df-9c5941dbc383","task_key":"default--validation--48165","task_revision_id":"1","upstream_id":"48165","short_description":"HellaSwag validation 48165","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to get nice nails [title] Stop biting your nails. [step] Biting your nails keeps them from growing strong. When you bite them off you're actually tearing the nail, which weakens it all the way to the nail bed.\",\"endings\":\"[\\\"The toenail grows back faster and stronger in its first few months, even though it's only stained for a short amount of time. Biting your nails at least two hours before bedtime is the best way to help preserve the nails.\\\", \\\"The saliva from your mouth also softens your nails, making them more prone to getting torn or broken. Biting your nails is a habit that's hard to break, but it's the first major step to having nice-looking nails! [substeps] You can buy products that will keep you from biting your nails by making them taste funny.\\\", 'If you do bite your nails, it breaks the nail bed, making it look like a ball has been dropped on it. [substeps] Use nail clippers to remove any loose pieces and dry off your nails after you have cleaned them.', 'So when you bite your nails, bite just a little bit off, just to keep your nails strong. [title] Consider taking a break during a manicure.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}