{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"abcd6cbf-aa53-5c8d-a609-ddfea41fef8b","task_key":"default--test--6459","task_revision_id":"1","upstream_id":"6459","short_description":"The basic code of ethical research (respect for persons, beneficence, and…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['World Medical Association (WMA)', 'United Nations Educational, Scientific and Cultural Organization (UNESCO)', 'Geneva Convention', 'World Health Organization (WHO)', 'Declaration of Helsinki', 'Nuremberg Code', 'CIOMS', 'International Committee of Medical Journal Editors (ICMJE)', 'Belmont Report', 'American Psychological Association (APA)']\",\"question\":\"The basic code of ethical research (respect for persons, beneficence, and justice) was first established by the:\",\"src\":\"ori_mmlu-virology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}