{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b4e0a5e2-9d59-55bc-82e2-2e81e151eecf","task_key":"default--test--3382","task_revision_id":"1","upstream_id":"3382","short_description":"A proficient engineer can easily design skeletal structures that are more…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['the skeletal structures of these mammals are not designed for functionality, but for other purposes.', 'these mammals have not evolved to their full potential yet.', 'natural selection has not had sufficient time to create the optimal design in each case, but will do so given enough time.', 'natural selection favors variety and diversity over optimal functionality.', 'the complexity of the skeletal system hinders the possibility of optimal arrangement.', 'natural selection is generally limited to modifying structures that were present in previous generations and in previous species.', 'the skeletal structures are indeed optimally arranged, but our understanding of functionality is limited.', 'optimal design is a human concept that does not apply to natural selection.', 'in many cases, phenotype is not merely determined by genotype, but by the environment as well.', 'natural selection operates in ways that are beyond the capability of the human mind to comprehend.']\",\"question\":\"A proficient engineer can easily design skeletal structures that are more functional than those currently found in the forelimbs of such diverse mammals as horses, whales, and bats. That the actual forelimbs of these mammals do not seem to be optimally arranged is because\",\"src\":\"ori_mmlu-high_school_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}