{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"df2416be-a614-5ecb-a1c2-818e361cfd72","task_key":"default--test--8632","task_revision_id":"1","upstream_id":"8632","short_description":"One model for the spread of an epidemic is that the rate of spread is jointly…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['22 days', '20 $\\\\\\\\mathrm{days}$', '12 days', '18 days', '8 days', '30 days', '25 $\\\\\\\\mathrm{days}$', '10 $\\\\\\\\mathrm{days}$', '15 $\\\\\\\\mathrm{days}$', '35 days']\",\"question\":\"One model for the spread of an epidemic is that the rate of spread is jointly proportional to the number of infected people and the number of uninfected people. In an isolated town of 5000 inhabitants, 160 people have a disease at the beginning of the week and 1200 have it at the end of the week. How long does it take for $80 \\\\%$ of the population to become infected?\",\"src\":\"scibench-calculus\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}