{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"af030fa3-16a1-5bfb-9ed8-786c449e465a","task_key":"default--test--7792","task_revision_id":"1","upstream_id":"7792","short_description":"Statement 1 | For every positive integer n there is a cyclic group of order n.…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['Statement 1 is false because not all positive integers have a cyclic group. Statement 2 is true.', 'Statement 1 is false because only composite numbers have a cyclic group. Statement 2 is true.', 'False, False', 'True, False', 'False, True', 'Statement 1 is false because only prime numbers have a cyclic group. Statement 2 is true.', 'Statement 1 is true because all positive integers have a cyclic group. Statement 2 is false because only infinite cyclic groups contain an element of every order that divides the order of the group.', 'True, True', 'Both statements are true, but they are not related.', 'Statement 1 is true because all positive integers have a cyclic group. Statement 2 is false because not all finite cyclic groups contain an element of every order that divides the order of the group.']\",\"question\":\"Statement 1 | For every positive integer n there is a cyclic group of order n. Statement 2 | Every finite cyclic group contains an element of every order that divides the order of the group.\",\"src\":\"ori_mmlu-abstract_algebra\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}