{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"hellaswag","formal_name":"HellaSwag","introduction":"HellaSwag asks which of four continuations naturally follows a sentence or a video caption. The wrong options are produced by adversarial filtering, so items that are nearly trivial for people remain hard for models.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://rowanzellers.com/hellaswag/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8472f079-66db-5707-a537-6b6910f50ecf","task_key":"default--validation--25469","task_revision_id":"1","upstream_id":"25469","short_description":"HellaSwag validation 25469","config":"default","split":"validation","body":"{\"activity_label\":\"Work World\",\"ctx\":\"[header] How to become an optometrist [title] Obtain a bachelor's degree in pre-med or science. [step] Before you can apply to an optometry program, you need to complete an undergraduate degree. Most optometry schools require coursework in biology, chemistry, physics, english, and mathematics.\",\"endings\":\"['A pre-med degree or degree in a biological science will help prepare you for med school. Get a well-rounded undergrad education and work hard to get good grades! Medical schools will want to see that you take your education seriously.', '[substeps] College degree requirements for enrolling in an accredited pre-med program varies by state. Ask your guidance counselor if this is a good choice for you.', \\\"Depending on your interests, you may also want to complete a bachelor's degree in political science, cultural studies, and business. [substeps] Pursue something like a master's degree or a doctoral degree (bfa).\\\", \\\"Then pass an exam which covers all four of these subjects and you will get a gpa. [substeps] An additional requirement might be having a master's degree in a specific subject, like biology.\\\"]\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://rowanzellers.com/hellaswag/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}