{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"dd51aa43-afdb-5e55-ad2e-24ca5ec7eec9","task_key":"default--test--2855","task_revision_id":"1","upstream_id":"2855","short_description":"Which of the following statements most detracts from the claim that the human…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['The size of the appendix varies greatly among individuals.', 'Individuals with a larger-than-average appendix leave fewer offspring than those with a below-average-sized appendix.', 'The appendix is located near the large intestine.', 'The appendix is not present in all mammal species.', 'The appendix can be surgically removed with no immediate ill effects.', 'The appendix might have been larger in fossil hominids.', 'The appendix has a substantial amount of defensive lymphatic tissue.', 'The appendix is often inflamed in cases of appendicitis.', \\\"Many people live healthy lives without an appendix after it's been removed.\\\", 'The appendix has no known function in digestion.']\",\"question\":\"Which of the following statements most detracts from the claim that the human appendix is a completely vestigial organ?\",\"src\":\"ori_mmlu-high_school_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}