{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"38d7dc49-7046-54a7-a7d2-282647673a0c","task_key":"default--test--6098","task_revision_id":"1","upstream_id":"6098","short_description":"Simple tandem repeat polymorphisms in humans are most useful for","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['studying the effects of radiation exposure', 'identifying the presence of bacterial infections', 'solving criminal and paternity cases', 'determining blood type', 'transferring disease resistance factors into bone marrow cells', 'estimating relationships of humans and Neanderthals', 'accelerating the process of aging', 'predicting the risk of genetic disorders', 'reconstructing the relationships of humans and chimps.', 'determining the sex of an unborn child']\",\"question\":\"Simple tandem repeat polymorphisms in humans are most useful for\",\"src\":\"ori_mmlu-medical_genetics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}