{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f5ccc33f-58ed-5bfb-90c5-8cc2dba87c1c","task_key":"default--validation--50512","task_revision_id":"1","upstream_id":"50512","short_description":"HellaSwag validation 50512","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to convince your parent to let you use tampons [title] Have courage. [step] You may be a little nervous about asking your guardian permission to use tampons. Especially if they've told you right off the bat that you should wait until you're older.\",\"endings\":\"[\\\"Know that it's okay to ask for permission, but be clear that you absolutely must do so. [substeps] While you may think that you should try it, don't overthink it.\\\", \\\"Plus, they may be surprised when they see you wearing tampons in public without their permission. It's your safety, not theirs.\\\", 'Bring it up when you\\\\'re the only ones in the kitchen, car, living room, bathroom, etc. Try saying such things as: \\\" hey mom, would you be okay with me using tampons as long as you could explain how? I will probably need help, but i think i\\\\'m ready.', \\\"But there's really no place to rush. Wait, and do what feels natural for you.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}