{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b1f4ea9e-aeba-52de-a051-3fad904333a5","task_key":"default--validation--16939","task_revision_id":"1","upstream_id":"16939","short_description":"HellaSwag validation 16939","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to choose a dog leash [title] Choose a nylon leash if you want variety. [step] Available in nearly every color of the rainbow, nylon leashes are also relatively inexpensive and strong. They are durable and good for rainy or humid environments.\",\"endings\":\"['Since they are lightweight and easy to handle, nylon leashes are good to use when training a puppy. [substeps] Nylon leashes can become slippery when wet, so do not use them while swimming with your dog.', \\\"Nylon leashes are also easy for a novice to tie. Nylon leashes are relatively inexpensive and highly adjustable, giving you the opportunity to roll all the way up the front of the dog's head from the forehead, down the face, and back to its tail.\\\", \\\"[substeps] Nylon leashes are generally more power-efficient, won't snag on trees and grass and will not actually injure dogs. They are also more stable, as they won't be knocked over or stepped on by a dog that is already larger or bigger.\\\", \\\"[title] Decide what kind of leash you want your dog to wear. [step] Typically, refers to a long leash that attaches to your dog's lead leg.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}