{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2fa46eb6-f361-559d-9077-4700aea1f257","task_key":"default--validation--34856","task_revision_id":"1","upstream_id":"34856","short_description":"HellaSwag validation 34856","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to explain autism to people [title] Know what the general definition of autism is. [step] Autism is a developmental disability that generally leads to differences in communication and social skills. It is a neurological difference that can present difficulties, but also advantages.\",\"endings\":\"['In the modern world, autism stems from differences in how people participate in social events. But even an instructive physical demonstration can help you understand the distinction between autism and other neurorelated mental disorders.', '[title] Learn what autistic people have to say about autism. [step] Autistic people, experiencing the differences and urges themselves, can offer the greatest insight into how autism works.', \\\"[substeps] Autism introduces a wide range of developmental problems to a person who is 18. Disability increases a person's emotional responses to situations they may not otherwise experience.\\\", 'Common autism issues with social skills include : [substeps] Behavioral deficits of both highly intelligent and highly educated students individuals affected by stereotypes or ridicule poor communication and conduct low morale or interpersonal relationships [title] Decide if your doctor values examining you. [step] Many doctors judge by physical appearance, while others are very concerned about the quality of their patients.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}