{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"15b5ce91-662f-5b04-b164-f7c66d7e2dd1","task_key":"all--test--15b5ce91-662f-5b04-b164-f7c66d7e2dd1","task_revision_id":"1","upstream_id":"","short_description":"The son of a bricklayer goes to college and i) becomes a teacher at a medical…","config":"all","split":"test","body":"{\"choices\":\"['intergenerational mobility with respect to the father, horizontal mobility, horizontal mobility', 'intragenerational mobility with respect to the son, upward mobility, upward mobility', 'intergenerational mobility with respect to the father, upward mobility, horizontal mobility', 'intragenerational mobility with respect to the son, horizontal mobility, upward mobility']\",\"question\":\"The son of a bricklayer goes to college and i) becomes a teacher at a medical school, ii) gets promoted to tenured professor, and iii) moves across the country for a new tenured professor position at a different school. Sequentially, this man has experienced:\",\"subject\":\"college_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}