{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0aaeab5c-865c-5076-85d2-d242e3674c6c","task_key":"default--test--6169","task_revision_id":"1","upstream_id":"6169","short_description":"When you examine a patient who has suffered an unfavourable fracture of the body…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['anterior fragment to be displaced anteriorly by the action of the lateral pterygoid muscle.', 'posterior fragment to be displaced downwards by the action of the digastric muscle.', 'anterior fragment to be displaced backwards by action of the temporalis muscle.', 'posterior fragment to be displaced laterally by the action of the medial pterygoid muscle.', 'anterior fragment to be displaced medially by the action of the medial pterygoid muscle.', 'posterior fragment to be displaced upwards by the action of the masseter muscle.', 'posterior fragment to be displaced anteriorly by the action of the lateral pterygoid muscle.', 'anterior fragment to be displaced upwards by the action of the masseter muscle.', 'anterior fragment to be displaced downwards by the action of the digastric muscle.', 'posterior fragment to be displaced medially by action of the medial pterygoid muscle.']\",\"question\":\"When you examine a patient who has suffered an unfavourable fracture of the body of the mandible, you would expect the\",\"src\":\"ori_mmlu-anatomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}