{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9a6127d2-aa69-5cee-8c33-98b753d36988","task_key":"default--test--8103","task_revision_id":"1","upstream_id":"8103","short_description":"Statement 1 | If T: V -> W is a linear transformation and dim(V ) < dim(W) < 1,…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['Neither True nor False, Neither True nor False', 'Neither True nor False, False', 'True, False', 'False, True', 'True, True but not always', 'Neither True nor False, True', 'True, Neither True nor False', 'False, Neither True nor False', 'False, False', 'True, True']\",\"question\":\"Statement 1 | If T: V -> W is a linear transformation and dim(V ) < dim(W) < 1, then T must be injective. Statement 2 | Let dim(V) = n and suppose that T: V -> V is linear. If T is injective, then it is a bijection.\",\"src\":\"ori_mmlu-abstract_algebra\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}