{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"463385ae-c555-5140-916d-c36b397bcb2d","task_key":"default--test--3120","task_revision_id":"1","upstream_id":"3120","short_description":"In garden peas, the allele for tall plants (D) is completely dominant to the…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['All dwarf white', '1 tall violet: 2 tall white: 1 dwarf white', '1 tall violet: 1 dwarf violet: 1 tall white: 1 dwarf white', 'All dwarf violet', '1 tall violet: 1 tall white: 1 dwarf violet: 1 dwarf white', 'All tall violet', '1 tall violet: 1 dwarf violet', '2 tall violet: 1 dwarf violet: 1 dwarf white', '1 tall violet: 1 dwarf white', '1 tall violet: 1 tall white']\",\"question\":\"In garden peas, the allele for tall plants (D) is completely dominant to the allele for dwarf plants (d) and the allele for violet flower color (W) is completely dominant to the allele for white flower color (w). In a cross between a tall violet plant, with the genotype DDWw, and a dwarf white plant, what phenotypic ratio of the progeny would be expected from this cross?\",\"src\":\"ori_mmlu-college_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}