{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ea68309e-1748-50af-9206-9538b17be15e","task_key":"default--test--8691","task_revision_id":"1","upstream_id":"8691","short_description":"Ms. Ling has been living in her home for exactly 4 years. She knows that there…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['12 ÷ m = 4', 'm ÷ 4 = 12', 'm × 4 = 12', '12 × m = 4', 'm - 12 = 4', '4 × m = 12', '4 + m = 12', '12 – m = 4', 'm - 4 = 12']\",\"question\":\"Ms. Ling has been living in her home for exactly 4 years. She knows that there are 12 months in each year. She found the total number of months, m, she has been living in her home by using the number sentence below. 4 × 12 = m Which number sentence shows another way to find the total number of months, m, Ms. Ling has been living in her home?\",\"src\":\"ori_mmlu-elementary_mathematics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}