{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b307ca50-5c0e-5e89-91ab-47b41ceded0e","task_key":"default--validation--41032","task_revision_id":"1","upstream_id":"41032","short_description":"HellaSwag validation 41032","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to care for your grandparents [title] Offer to help. [step] Ask how you can help out. If they're struggling to do something, step in and offer to take over.\",\"endings\":\"['Don\\\\'t walk into their room and say, \\\" i noticed you have a spare bedroom for mom and dad to use, \\\" or, \\\" i noticed you have a small cat. Would you like me to help you open a garage door? \\\" be aware that some grandparents may refuse anything you offer to them.', \\\"This shows your grandparents that you're interested in helping them with things like easy-going chores or completing their laundry, and that you like and care for them in general. [substeps] Get in the habit of turning down any assignments your grandparents want done.\\\", \\\"Even if your grandparents don't ask, take the initiative to offer up your help. You can even ask your parents if there's something you can do to help.\\\", \\\"Let them know you're there for them and offer your time and attention. Don't just offer to take care of them.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}