{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ec2f9e96-407e-53f9-bafb-b50ba7b01cbc","task_key":"default--test--2939","task_revision_id":"1","upstream_id":"2939","short_description":"Which of the following depicts the correct sequence of membranes of the…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Outer membrane, inner membrane, thylakoid membrane', 'Thylakoid membrane, inner membrane, outer membrane', 'Inner membrane, thylakoid membrane, outer membrane', 'Thylakoid membrane, outer membrane, inner membrane', 'Strama, outer membrane, inner membrane', 'Inner membrane, strama, outer membrane', 'Inner membrane, outer membrane, thylakoid membrane', 'Outer membrane, thylakoid membrane, inner membrane', 'Strama, thylakoid membrane, outer membrane', 'Outer membrane, strama, thylakoid membrane']\",\"question\":\"Which of the following depicts the correct sequence of membranes of the chloroplast, beginning with the innermost membrane and ending with the outermost membrane?\",\"src\":\"ori_mmlu-college_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}