{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"02b26ba2-ef19-508f-9d62-25fb47d98807","task_key":"default--test--6424","task_revision_id":"1","upstream_id":"6424","short_description":"In a patient with a normal healthy mouth, you would expect the mucosa covering…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['light pink below the mucogingival junction and red above it.', 'Dark pink below the mucogingival junction and red above it.', 'red below the mucogingival junction and light pink above it.', 'White below the mucogingival junction and red above it.', 'Light pink below the mucogingival junction and white above it.', 'light pink in colour on both sides of the mucogingigival junction.', 'Dark pink on both sides of the mucogingival junction.', 'red on both sides of the mucogingival junction.', 'Dark pink below the mucogingival junction and light pink above it.']\",\"question\":\"In a patient with a normal healthy mouth, you would expect the mucosa covering the alveolar process supporting the mandibular teeth to be\",\"src\":\"ori_mmlu-anatomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}