{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"23a6a4cf-bfee-5f17-9a6a-bc4cf94e502c","task_key":"default--test--7479","task_revision_id":"1","upstream_id":"7479","short_description":"What institution is closely identified with capitalism,perhaps moreclosely than…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['The institution of communal property', 'The institution of cooperative ownership', 'The institution of intellectual property', 'The institution of social welfare', 'The institution of state property', 'The institution of market regulation', 'The institution of public property', 'The institution of collective bargaining', 'The institution of inheritance rights', 'The institution of private property']\",\"question\":\"What institution is closely identified with capitalism,perhaps moreclosely than any other?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}