{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5292b12c-2f2d-539c-aadc-3ba2b525fbb6","task_key":"default--validation--45768","task_revision_id":"1","upstream_id":"45768","short_description":"HellaSwag validation 45768","config":"default","split":"validation","body":"{\"activity_label\":\"Health\",\"ctx\":\"[header] How to become a pulmonologist [title] Take the right classes in high school. [step] While you are in high school, make sure to focus on getting a strong science background. This will set you up for success in your science classes in college.\",\"endings\":\"['It will also help you get acquainted with the world more thoroughly. [substeps] Trigonometry, biology, and chemistry are the four most important courses to learn in college.', 'Attending a high school spring semester or summer school can make you a stronger candidate for future career success. [substeps] Pass the mayo clinic examination in chemistry (learn more about mixer or colon) and creative writing in math and middle school.', 'Make sure to take biology, physiology, anatomy, and chemistry, if your school offers them. [substeps] Although not as necessary as your science background, it may help to have a strong background in math, with focus on geometry and algebra.', '[substeps] Talk to your teachers, teachers, and supervisors about how good you are at taking the field. Keep in mind that your degree will not assist in your career choice unless you can get a teaching certification.']\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}