{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"318a99df-472e-5a4b-bc4c-7c84d5d5359b","task_key":"default--test--6604","task_revision_id":"1","upstream_id":"6604","short_description":"You notice a lesion on the labial alveolar mucosa of the lower right lateral…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['The mental nerve', 'The hypoglossal nerve', 'The incisive nerve', 'The buccal nerve', 'The inferior alveolar nerve', 'The infraorbital nerve', 'The facial nerve', 'The maxillary nerve', 'The lingual nerve', 'The trigeminal nerve']\",\"question\":\"You notice a lesion on the labial alveolar mucosa of the lower right lateral incisor tooth of one of your patients and decide to take a biopsy to send for oral pathology report Which nerve would require local anaesthesia in order to carry out a pain-free biopsy?\",\"src\":\"ori_mmlu-anatomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}