{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2e707a69-ba1b-5d13-a22e-5402409e84ff","task_key":"default--validation--21723","task_revision_id":"1","upstream_id":"21723","short_description":"HellaSwag validation 21723","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to participate in class [title] Develop a relationship with your teacher. [step] Make a point to introduce yourself to your teacher before class starts. This is a great way to make a good impression.\",\"endings\":\"['[substeps] When you sit next to your teacher, try to make eye contact and give a nod or some kind of verbal acknowledgement. Doing this will put your teacher at ease as well.', 'The advantage of making a personal connection is that you will have the time to sit down one on one. You will both have the opportunity to talk and stand up for the class, take notes, and discuss your experiences.', 'Introducing yourself is especially important if you are in a large class, such as a lecture with hundreds of other students, as your teacher may never even learn your name otherwise. It shows your teacher that you care about the class and are willing to put in effort.', \\\"Even if you don't know the teacher, you can say hello and ask how they are doing. Remind him or her of the skills you want to learn in class so they can teach you.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}