{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"46019f50-26ae-50c5-9358-6bbd52dec601","task_key":"default--validation--40733","task_revision_id":"1","upstream_id":"40733","short_description":"HellaSwag validation 40733","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to boycott autism speaks [title] Consider your mental health. [step] If you have disabilities such as anxiety, depression, ptsd, or other mental health conditions, reading about as may trigger a downward spiral. If you are prone to this, proceed with caution.\",\"endings\":\"['[substeps] Autistic people with self-esteem issues should especially be mindful. Reading what autism speaks has to say may worsen your self-image.', 'Read stories or articles on the autism speaks website and read for relief from disease. Avoiding the internet can be a good start to budgeting money for home clothes.', \\\"Even if you can't see, sound in depth about your needs and symptoms. [substeps] Make sure any adult can see what the real problem is so that it doesn't become more severe.\\\", \\\"Try to unfollow autism speaks books and read nonfiction books that showcase that topic, whether you are autistic or not. Don't worry: many successful writers have found friends who actively listen to their own thoughts.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}