{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ead6136b-3251-5946-92f1-a296e95e2cea","task_key":"default--test--6422","task_revision_id":"1","upstream_id":"6422","short_description":"Which muscle is the most active during a right lateral excursion of the…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Left masseter muscle', 'Right buccinator muscle', 'Right temporalis muscle', 'Right masseter muscle', 'Right lateral pterygoid muscle', 'Left medial pterygoid muscle', 'Left temporalis muscle', 'Left buccinator muscle', 'Left lateral pterygoid muscle', 'Right medial pterygoid muscle']\",\"question\":\"Which muscle is the most active during a right lateral excursion of the mandible?\",\"src\":\"ori_mmlu-anatomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}