{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c3da898b-8cb3-5a01-aca0-15db021b3847","task_key":"default--validation--18601","task_revision_id":"1","upstream_id":"18601","short_description":"HellaSwag validation 18601","config":"default","split":"validation","body":"{\"activity_label\":\"Personal Care and Style\",\"ctx\":\"[header] How to be a good tutor [title] Pay attention to what they already know. [step] When first getting to know your students, you need to gauge what they already know, so you don't waste time in your sessions. Ask the student what they're good at and what they most enjoy about the subject you're working on.\",\"endings\":\"['If you have good sources of information, these should be the parts they know. [substeps] There are many ways to apply knowledge.', \\\"You can use your knowledge to decide what they're good at. For example, if they love math, ask them if they can explain diagrams.\\\", \\\"Here's how to do it : [substeps] Evaluate the student's interests by answering any questions related to the topic they're studying. Organize a class discussion and find as much information as you can.\\\", \\\"Let him or her speak open-endedly about the subject and show off for you. It will make the student feel smart and validated while letting you figure out what material they've already mastered.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}