{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c1d6786f-8400-5ba1-89b8-0cfcaa84deb1","task_key":"all--test--c1d6786f-8400-5ba1-89b8-0cfcaa84deb1","task_revision_id":"1","upstream_id":"","short_description":"What does the presence of discouraged workers do to the measurement of the…","config":"all","split":"test","body":"{\"choices\":\"['Discouraged workers are counted as \\\"out of the labor force\\\" thus the unemployment rate is understated making the economy look stronger than it is.', 'Discouraged workers are counted as \\\"out of the labor force\\\" thus the unemployment rate is understated making the economy look weaker than it is.', 'Discouraged workers are not surveyed so there is no impact on the unemployment rate.', 'Discouraged workers are counted as \\\"unemployed\\\" thus the unemployment rate is understated making the economy look stronger than it is.']\",\"question\":\"What does the presence of discouraged workers do to the measurement of the unemployment rate?\",\"subject\":\"high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}