{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4d2ab636-e7c7-5445-b5ac-a721acbf107a","task_key":"default--validation--12365","task_revision_id":"1","upstream_id":"12365","short_description":"HellaSwag validation 12365","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to be confident when reading out loud in class [title] Know you'll be called upon to read aloud at some point. [step] Written language is the best system humans have ever devised for transmitting knowledge. Naturally, your teachers will want you to show what you've learned.\",\"endings\":\"[\\\"Be proud of your accomplishments and include a pen and some notes as a reward later. [substeps] In addition to being interesting to your teacher, you'll want to inspire your classmates with their lessons.\\\", \\\"Why is this for you? [substeps] You'll probably give examples yourself. In this case, it's because you know you'll be called upon to work on a project and you have the time.\\\", 'In english and social studies classes, you do most of your learning by reading-so these are the most common classes to read aloud in. [substeps] However, you can be asked to read something for the class in any subject, even gym.', \\\"You should know when you'll get called to class that you will be called to stop talking while you read. [title] Pay attention to the atmosphere and posture of the class.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}