{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e9a54984-a2d1-5bc2-a7bd-99fc89e56c60","task_key":"default--validation--49315","task_revision_id":"1","upstream_id":"49315","short_description":"HellaSwag validation 49315","config":"default","split":"validation","body":"{\"activity_label\":\"Food and Entertaining\",\"ctx\":\"[header] How to get more disabled friends [title] Try support groups for people with disabilities. [step] There are support groups for specific disabilities (e.g. cerebral palsy), groups of disabilities (e.g. mental illnesses), or disability in general. [substeps] Make sure that you qualify for the group before you go.\",\"endings\":\"['Many people join groups in order to feel supported. Meet with others who have disabilities or those in your support group to see if you qualify.', 'For example, if the group is only for people who have down syndrome, and you are the mother of a child with down syndrome, this is not a space for you. [title] Look for disability-related clubs and activities.', \\\"Contact the group's lawyer and ask for an application. You can also find support groups online at [https: // www.clinic.\\\", \\\"Because some groups are sponsored by social services or adult literacy organizations, you may not be eligible for one. Just don't go into these groups knowing that you will be raising support for the disability, as it might hurt you.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}