{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"13f6469f-0cec-51fc-94ad-5609f65f24b7","task_key":"default--validation--14163","task_revision_id":"1","upstream_id":"14163","short_description":"HellaSwag validation 14163","config":"default","split":"validation","body":"{\"activity_label\":\"Education and Communications\",\"ctx\":\"[header] How to succeed in distance learning [title] Familiarize yourself with the syllabus. [step] Think of your syllabus as the blueprint for your class. It should contain all of the class guidelines and rules, as well as what your instructor expects from you over the course of the semester.\",\"endings\":\"[\\\"If you're having trouble understanding all of the syllabus instructions, ask your instructor for advice. [substeps] To become a successful athlete, you have to have a strong understanding of the game and field, and then, with that knowledge, bring out your best on a single outing.\\\", \\\"Include these with your notes, papers, assignments and applications so you'll be up to date and able to fully experience your classroom. [substeps] Do a quick background check on your syllabus for your instructor.\\\", 'Bring this information to the syllabus, stating all required daily requirements, as well as any tests you may have scheduled or deadlines you must meet for your session. This way, you can easily adjust any assignments you may finish before the next session starts.', 'Make sure you read through this document in its entirety and keep track of it all semester. [substeps] Ask your instructor any questions you might have about the syllabus.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}