{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a5f1d824-27eb-5ca3-bcd1-bd37d9b5ff3e","task_key":"default--validation--4012","task_revision_id":"1","upstream_id":"4012","short_description":"HellaSwag validation 4012","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to get organized for college or grad school [title] Read through your syllabi. [step] Each professor should provide you with a syllabus on the first day of class. The syllabus will list all the materials you need for each class in addition to a schedule of assignments with due dates.\",\"endings\":\"[\\\"[title] Confirm any assignments you need to do for upcoming classes. [step] If you haven't assigned a class on a given day, you should call the professor to confirm.\\\", \\\"Look over the syllabus and make sure you are on track. Include the syllabi on these papers so you don't forget.\\\", \\\"[substeps] Look up section headings and pages for upcoming quizzes and assignments, and when they're due meet up with teachers if you have the time. Some universities also publish a class schedule template in their website.\\\", 'Carefully read over each syllabus as soon as you get it. Highlight the materials you need as well as other pertinent information, such as attendance and late work policies.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}