{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8c7f3b07-3025-50c6-b989-63c6b69d1a49","task_key":"default--test--6486","task_revision_id":"1","upstream_id":"6486","short_description":"Which of the following muscles are derived from the first brachial arch?","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['The muscles of mastication', 'The muscles of mastication, anterior and posterior bellies of digastric and geniohyoid', 'The muscles of mastication, anterior belly of digastric and stylohyoid', 'The muscles of mastication, anterior belly of digastric and mylohyoid', 'The muscles of mastication, anterior and posterior bellies of digastric and mylohyoid', 'The muscles of mastication and mylohyoid', 'The muscles of mastication and anterior belly of digastric', 'The muscles of mastication, anterior belly of digastric, geniohyoid and mylohyoid', 'The muscles of mastication and anterior and posterior bellies of digastric', 'The muscles of mastication, anterior and posterior bellies of digastric, geniohyoid and mylohyoid']\",\"question\":\"Which of the following muscles are derived from the first brachial arch?\",\"src\":\"ori_mmlu-anatomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}