{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"23a0c237-fe99-50b6-b50e-4510cd35c00d","task_key":"default--test--2810","task_revision_id":"1","upstream_id":"2810","short_description":"The theory of evolution is most accurately described as","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['a universally accepted fact about the origin and development of all species.', 'a hypothesis in the process of being tested and verified.', 'a disproven theory about the development of species over time.', 'a speculation about possible changes in populations over time.', 'an opinion that some scientists hold about how living things change over time.', 'an overarching explanation, supported by much evidence, for how populations change over time.', 'one possible explanation, among several scientific alternatives, about how species have come into existence.', 'an educated guess about how species originate.', 'an assumption based on observations of a few species.', 'a religious belief about the origin of species.']\",\"question\":\"The theory of evolution is most accurately described as\",\"src\":\"ori_mmlu-high_school_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}