{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"55737573-2375-5ca1-b22a-de0f45bf32eb","task_key":"default--validation--13082","task_revision_id":"1","upstream_id":"13082","short_description":"HellaSwag validation 13082","config":"default","split":"validation","body":"{\"activity_label\":\"Pets and Animals\",\"ctx\":\"[header] How to train a dwarf hamster to recognize its name [title] Use the name frequently. [step] Whenever you are around your hamster, use its name. Handle your dwarf hamster every day too, to ensure the trust between you and your pet.\",\"endings\":\"[\\\"To handle the hamster, use your fingers to get near it's eye, and don't touch the hamster's nose. As time goes on, you can start treating your dwarf hamster with more fun treats.\\\", 'This also keeps your pet stimulated and healthy. [title] Use a treat to solidify the connection with its name.', \\\"If it hears the name or sees you, the hamster may become alarmed and begin to fear you. [title] Use your dwarf hamster's name every time you pet it.\\\", '[substeps] Let your hamster know that you are there to say \\\" hello \\\" and let it know you have come to its rescue. [title] Hold your dwarf hamster before you teach it to play.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}