{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2b52f833-f274-5db0-888e-cbc2b1b55bb6","task_key":"default--validation--38302","task_revision_id":"1","upstream_id":"38302","short_description":"HellaSwag validation 38302","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to apologize to a teacher [title] Reflect on what you did wrong. [step] In order to make your apology sincere, you have to actually understand where you were at fault. There is a difference between being sorry you made your teacher mad and being sorry you did not do the homework that made your teacher mad.\",\"endings\":\"['When you are just starting off, try to think of both. Consider how you contributed to the problem the first time around and what you did differently next time.', 'For example, while your teacher may still be mad at you when you missed to clean up the mess, you did not do homework. [substeps] Be honest about what you did wrong so that you do not permanently hurt your teacher or have an argument.', \\\"While you shouldn't expect to change your behavior and feel bad about it, you should understand how to be upfront and willing to make amends. [substeps] For example, suppose you made a mistake on your algebra homework.\\\", \\\"Understanding what you did wrong will help you figure out what to say to your teacher when you do apologize. [substeps] For example, you'd say one thing when apologizing for being rowdy in class, and you'd say a different thing when apologizing for forgetting your homework.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}