{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"45f6a463-191e-5740-b8b4-a4669ed20875","task_key":"default--test--5518","task_revision_id":"1","upstream_id":"5518","short_description":"A sect is organised around:","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['belief in the afterlife, individual interpretation of scripture, and weekly gatherings', 'beliefs that can be challenged by members, a charismatic leader, and openness to new recruits', 'hierarchical structure, a pope as the head, and a closed canon of scripture', 'strict rules, a single leader, and exclusion of non-members', 'tolerance towards other religions and the separation of church and state', 'acceptance of all beliefs, lack of leadership, and focus on personal growth', 'worship of a specific deity, regular rituals, and mandatory tithes', 'voluntary membership, democratic leadership, and high levels of emotional commitment', 'the idea of the secular world itself as being sacred', 'a single holy book, a rigid moral code, and a closed community']\",\"question\":\"A sect is organised around:\",\"src\":\"ori_mmlu-sociology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}