{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3a072810-5f37-527c-acc7-6c35c98b8b24","task_key":"default--validation--42981","task_revision_id":"1","upstream_id":"42981","short_description":"HellaSwag validation 42981","config":"default","split":"validation","body":"{\"activity_label\":\"Youth\",\"ctx\":\"[header] How to help your sibling with homework [title] Find out what the exact assignment is. [step] Read through the assignment and make sure that you understand the material and are confident helping your sibling with their homework. It's important that you know the subject matter and type of questions before you start helping.\",\"endings\":\"['[substeps] At some schools, parents may provide clarification, but sometimes they will provide explanation of your homework and instructions as well. Ask your sibling what they will be working on and in what ways.', 'It is really important that you know and understand this topic so that it is less of a burden. [substeps] If you have to, be prepared to share this information with a parent or a guidance counselor.', 'If the answer is easy to follow, then that is fine. If you can just repeat the answer at home to check for spelling or grammar errors, then it might be easier for your sibling to pay attention and to feel understood without any work.', 'Your sibling may not have understood the assignment for themselves, or may be confused about how to follow the homework rules. If you help them understand the assignment, they may not need further help.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}