{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ce47710c-1923-5f07-a319-166ba05cf578","task_key":"default--test--11183","task_revision_id":"1","upstream_id":"11183","short_description":"Rawls claims that students with fewer native assets (such as intelligence)…","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['the same level of attention and resources as those with more native assets.', 'educational resources based on their social background.', 'resources based on their willingness to learn.', 'educational resources based on their economic background.', 'less attention and fewer resources than those with more native assets.', 'entirely different educational resources than those with more native assets.', 'virtually no educational resources.', 'the opportunity to compete with those having more native assets.', 'more attention and resources than those with more native assets.', 'access to only basic education.']\",\"question\":\"Rawls claims that students with fewer native assets (such as intelligence) should be given:\",\"src\":\"ori_mmlu-philosophy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}