{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fbdd083f-0b56-591b-a9ea-4d4531b7210e","task_key":"default--validation--40271","task_revision_id":"1","upstream_id":"40271","short_description":"HellaSwag validation 40271","config":"default","split":"validation","body":"{\"activity_label\":\"Relationships\",\"ctx\":\"[header] How to discuss transgender issues with a child [title] Use trans-inclusive language from the beginning. [step] This will help lay the foundation for an inclusive attitude, and make later talks easier. [substeps] \\\" most women can get pregnant.\",\"endings\":\"['\\\" \\\" most children can have sex with transgender people! \\\" is a contested topic. \\\" some women will have a hysterectomy because they have to see a doctor.', 'A few men and non-binary people can get pregnant too. \\\" \\\" most boys can go to the bathroom standing up like you.', '\\\" \\\" most babies are healthy and are safe. \\\" some trans-inclusive language is best understood in a conversation about transgender rights.', 'Every female may get pregnant by age 35. \\\" this brings us into conversation on the topic of trans-inclusive speech and culture so that we can figure out what kind of topic the child will be facing at that age.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}