{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c10ea9ac-2a3f-5ae8-ad26-f503fee59af7","task_key":"default--test--3212","task_revision_id":"1","upstream_id":"3212","short_description":"Which is a true statement concerning genetic variation?","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['It is created by the direct action of natural selection.', 'It tends to be reduced by the processes involved when diploid organisms produce gametes.', 'It must be present in a population before natural selection can act upon the population.', 'It is always increased by the processes involved when diploid organisms produce gametes.', 'It is not affected by the environmental changes.', 'It can only be created by artificial selection.', 'It is only present in a population after natural selection has acted upon the population.', 'It arises independently of changes in the environment.', 'It is always reduced by the processes involved when diploid organisms produce gametes.', 'It arises in response to changes in the environment.']\",\"question\":\"Which is a true statement concerning genetic variation?\",\"src\":\"ori_mmlu-high_school_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}