{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f30ac657-3677-5716-8b93-3560fc3d61b2","task_key":"default--validation--45566","task_revision_id":"1","upstream_id":"45566","short_description":"HellaSwag validation 45566","config":"default","split":"validation","body":"{\"activity_label\":\"Work World\",\"ctx\":\"[header] How to become a health teacher [title] Figure out which level you want to teach. [step] You will need to decide between elementary, middle school, and high school. Think about whether you like working with the smallest children, or whether you want the older students and more complicated discussions.\",\"endings\":\"['If you just like working with the most small kids, then you may want to go for the latter. [title] Look for an entry level teacher in your field.', 'Whichever level you choose, the students or people involved in the discussion will need to be adults. [substeps] For example, if you prefer working with people in high school, consider going into middle school.', 'Each level has a different set of requirements so you should decide as early as possible. [title] Determine a second area of interest for your degree.', \\\"[substeps] It's a good idea to start with second graders who have grades above the required level. There is a good chance they will pass the gap and have a serious problem, and that would hurt the chances of you teaching them the harder concepts.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}