{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7e9e14af-8ed7-5da2-b826-241c86e906d4","task_key":"all--test--7e9e14af-8ed7-5da2-b826-241c86e906d4","task_revision_id":"1","upstream_id":"","short_description":"Confounding is a particular challenge in nutritional epidemiology because","config":"all","split":"test","body":"{\"choices\":\"['people change their diets over time.', \\\"it is difficult to measure people's diets accurately in large studies.\\\", 'the are no good methods to adjust for confounding in nutritional studies.', 'different dietary components are correlated with each other, making it difficult to separate their effects.']\",\"question\":\"Confounding is a particular challenge in nutritional epidemiology because\\n\",\"subject\":\"nutrition\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}