{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bde46bd7-a295-5d51-8ed9-b6ef89c6a69c","task_key":"default--test--8028","task_revision_id":"1","upstream_id":"8028","short_description":"The first number in a number pattern is 28. The pattern rule is to add 14 to get…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['All the numbers in the pattern can be divided equally by 8.', 'All the numbers in the pattern can be divided equally by 10.', 'All the numbers in the pattern can be divided equally by 3.', 'All the numbers in the pattern can be divided equally by 14.', 'All the numbers in the pattern can be divided equally by 6.', 'All the numbers in the pattern can be divided equally by 4.', 'All the numbers in the pattern can be divided equally by 5.', 'All the numbers in the pattern can be divided equally by 9.', 'All the numbers in the pattern can be divided equally by 7.']\",\"question\":\"The first number in a number pattern is 28. The pattern rule is to add 14 to get the next number in the pattern. If the pattern continues, which statement is true?\",\"src\":\"ori_mmlu-elementary_mathematics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}