{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1d648751-11a7-5710-adb5-1642db6ba405","task_key":"default--test--6429","task_revision_id":"1","upstream_id":"6429","short_description":"Which of the following structures should be visible on a dental panoramic…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Only the mandibular canal and hard palate', 'The mandibular canal, maxillary sinus, hard palate, zygomatic arch, and ethmoid sinus', 'The mandibular canal, maxillary sinus, hard palate, zygomatic arch, and frontal sinus', 'The mandibular canal and maxillary sinus', 'The mandibular canal', 'The mandibular canal, maxillary sinus, hard palate, zygomatic arch, and middle ear', 'The mandibular canal, maxillary sinus, hard palate and zygomatic arch', 'The mandibular canal, maxillary sinus and hard palate', 'The mandibular canal, maxillary sinus, hard palate, zygomatic arch, and nasal cavity', 'The mandibular canal, maxillary sinus, and temporomandibular joint']\",\"question\":\"Which of the following structures should be visible on a dental panoramic radiograph?\",\"src\":\"ori_mmlu-anatomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}