{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"84e829b2-ab82-50b4-b9df-c4c56695235b","task_key":"default--validation--22496","task_revision_id":"1","upstream_id":"22496","short_description":"HellaSwag validation 22496","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to be emo in middle school [title] Learn what it means to be emo. [step] Before you can become emo, you have to understand exactly what it means. \\\" emo \\\" is short for emotive, and people who consider themselves to be emo are usually in touch with their emotions.\",\"endings\":\"[\\\"They don't feel the need to hide how they feel, even when they are sad or angry. [substeps] While music, clothing, hair, and makeup can play a large role in being emo, being comfortable with your feelings and expressing them is the most important part.\\\", \\\"This is an important step to emo, so have a good understanding of your gender. You don't have to have what everyone else has.\\\", 'You don\\\\'t need a dictionary dictionary and stick with the \\\" emotive \\\". Instead, learn what emotive means, like \\\" i am \\\", \\\" i am skinny \\\", \\\" i am strong and strong \\\", \\\" i\\\\'m confident in my body \\\", etc.', \\\"You can't be emo without being emo. [substeps] If you're emo, you want an emo vibe that's unique to you.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}