{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"090fd762-75fd-5592-96e8-f58275f4c85f","task_key":"default--test--5214","task_revision_id":"1","upstream_id":"5214","short_description":"Religious organizations such as the Church of Norway, Islam, the Church of…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['They are primarily based in Europe.', 'They are monotheistic.', 'They all practice infant baptism.', 'They all have a hierarchical church structure.', 'They are cultures as well as churches.', 'They believe in reincarnation.', 'They all observe the Sabbath on Saturday.', 'They prohibit the use of religious icons.', 'They practice separation of church and state.', 'They exclude women as clergy.']\",\"question\":\"Religious organizations such as the Church of Norway, Islam, the Church of England, and the Church of Greece have which of the following characteristics in common?\",\"src\":\"ori_mmlu-sociology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}