{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c40335db-3afd-5752-aa0c-9052e195a471","task_key":"default--validation--16205","task_revision_id":"1","upstream_id":"16205","short_description":"HellaSwag validation 16205","config":"default","split":"validation","body":"{\"activity_label\":\"Family Life\",\"ctx\":\"[header] How to celebrate being an empty nester [title] Take pride in your accomplishment. [step] Raising children is an enormous, life-defining and intense job. Now is the time to revel in the fact that you \\\" made \\\" it and produced an independent child (or children), ready to taken on the challenges of the world.\",\"endings\":\"[\\\"Know that this high level of achievement is not your fault. You're the same person you were before you started being an empty nester.\\\", \\\"Celebrate what you have achieved, what you can accomplish, and what you are capable of. [substeps] If you feel like you're all alone, make little one your toy.\\\", 'You have now entered a group of human beings who have made an enormous contribution to society by responsibly and consistently raising children to become fine young adults. [substeps] Think about what your child has accomplished.', 'No one wants the shock when the child who is \\\" new \\\" is born. [title] Don\\\\'t skimp on your passions.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}