{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ad49cc2c-506d-54dd-b6bb-8f2d066a7a53","task_key":"default--test--5301","task_revision_id":"1","upstream_id":"5301","short_description":"An anatomical difference between males and females is:","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['females have larger hearts than males', 'males have many times the number of nerve endings in the penis that females have in the clitoris.', 'males have an external sexual organ that gets engorged with blood, whereas females do not', 'males have one exit structure for urine and reproductive fluids, whereas females have two', 'males have a higher number of red blood cells than females', 'males have ovaries, whereas females do not', 'females have more chromosomes than males', 'females have a larger brain size than males', 'males have a shorter lifespan than females', 'all of the above']\",\"question\":\"An anatomical difference between males and females is:\",\"src\":\"ori_mmlu-human_sexuality\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}