{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"dd79ad60-64db-52c6-9747-3802a9e6e758","task_key":"default--test--6172","task_revision_id":"1","upstream_id":"6172","short_description":"Which of the following options completes the following statement? \"The right…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['is a semilunar valve', 'is responsible for pumping blood to the rest of the body.', 'is located between the left atrium and the left ventricle.', 'is prevented from everting by papillary muscles.', 'is open during systole.', 'prevents blood returning from the pulmonary trunk as the heart relaxes.', 'is the largest valve of the heart.', 'controls the flow of blood from the left ventricle to the right atrium.']\",\"question\":\"Which of the following options completes the following statement? \\\"The right atrioventricular valve…\\\"\",\"src\":\"ori_mmlu-anatomy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}