{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"652e61f6-e216-5a12-b1b6-53cda4249698","task_key":"default--test--11136","task_revision_id":"1","upstream_id":"11136","short_description":"On McGregor's view, our rights carve out","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['the boundaries of our moral obligations.', 'the domain of our personal identity.', 'all of the above.', 'what we are free to do.', 'the responsibilities we have towards others.', 'the scope of our professional roles.', 'the domain of our community.', 'none of the above.', 'the limits of our legal obligations.', 'the extent of our individual freedom.']\",\"question\":\" On McGregor's view, our rights carve out\",\"src\":\"ori_mmlu-moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}