{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a3e280dc-98bd-5072-8326-83df78e97808","task_key":"default--test--6355","task_revision_id":"1","upstream_id":"6355","short_description":"During an inferior alveolar nerve block the needle ideally passes","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['through medial pterygoid.', 'inferior to medial pterygoid.', 'superior and lateral to medial pterygoid.', 'anterior and lateral to medial pterygoid.', 'through lateral pterygoid.', 'posterior to medial pterygoid.', 'superior to medial pterygoid.', 'anterior and medial to medial pterygoid.', 'posterior and medial to medial pterygoid.', 'posterior and lateral to medial pterygoid.']\",\"question\":\"During an inferior alveolar nerve block the needle ideally passes\",\"src\":\"ori_mmlu-anatomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}