{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"af381412-1c64-5ace-9ec4-38ab7a54310c","task_key":"default--test--6426","task_revision_id":"1","upstream_id":"6426","short_description":"The lateral pterygoid muscle","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['is attached to the condylar process and elevates the mandible.', 'is attached to the coronoid process and retrudes the mandible.', 'is not attached to the mandible but helps in elevation.', 'is attached to the condylar process and protrudes the mandible.', 'is attached to the coronoid process and elevates the mandible.', 'is not attached to the mandible but helps in protrusion.', 'is attached to the coronoid process and depresses the mandible.', 'is attached to the condylar process and retrudes the mandible.', 'is attached to the condylar process and depresses the mandible.', 'is attached to the coronoid process and protrudes the mandible.']\",\"question\":\"The lateral pterygoid muscle\",\"src\":\"ori_mmlu-anatomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}