{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8e1720e6-e8f6-59ed-bf43-e185e00eb97a","task_key":"default--test--8577","task_revision_id":"1","upstream_id":"8577","short_description":"Statement 1 | f : X → Y is continuous and X is compact. f must be uniformly…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['Not applicable, Not applicable', 'True, False', 'Both statements are applicable, but cannot be definitively classified as true or false.', 'False, Not applicable', 'False, True', 'True, Not applicable', 'False, False', 'Not applicable, True', 'Not applicable, False', 'True, True']\",\"question\":\"Statement 1 | f : X → Y is continuous and X is compact. f must be uniformly continuous. Statement 2 | If f(x), g(x) : R → R are everywhere differentiable f(g(x)) must be everywhere differentiable.\",\"src\":\"ori_mmlu-college_mathematics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}