{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"06c94844-e436-5272-aab4-052e5b3f4c74","task_key":"default--test--5298","task_revision_id":"1","upstream_id":"5298","short_description":"Which of the following was not a strategy used by the Conservative government of…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['making secondary action (in support of workers elsewhere) illegal', 'enforcing mandatory arbitration in all labor disputes', \\\"banning political strikes that were not primarily concerned with the worker's own conditions of work\\\", 'imposing restrictions on picketing', 'implementing a national minimum wage', 'making all strike action illegal', 'outlawing collective bargaining', \\\"replacing the labour movement with government-controlled worker's councils\\\", 'privatizing all industries']\",\"question\":\"Which of the following was not a strategy used by the Conservative government of 1979 to reduce the power of the labour movement?\",\"src\":\"ori_mmlu-sociology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}