{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7ff581dd-7178-5c25-a47b-071403298ded","task_key":"default--validation--10689","task_revision_id":"1","upstream_id":"10689","short_description":"HellaSwag validation 10689","config":"default","split":"validation","body":"{\"activity_label\":\"Work World\",\"ctx\":\"[header] How to be a successful engineer [title] Prepare early. [step] You should try to get an early start if your goal is to be an engineer, during middle school and high school if you can. Engineers need a grounding in math and science, so plan your schedule around these subjects.\",\"endings\":\"['Load up on the right classes and enroll in activities that will give you an edge when college admissions roll around. [substeps] Make sure to take basic math courses like algebra i & ii, geometry, trigonometry, pre-calculus, and calculus.', 'Take classes in science and engineering if you can, or a few classes in math if you can, and do your homework early on. [title] Take classes in at least 3 majors.', 'Do this ahead of time and follow the curriculum. You may need to sit for the classes, not algebra and physics.', \\\"[substeps] Also, if you don't have time to study and don't have enough time to sit down to study, try to keep an organized worksheet. You should print off those worksheets once you are done with school.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}