{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"dfa78dd8-864c-54f8-8462-04d139c788fd","task_key":"default--test--3306","task_revision_id":"1","upstream_id":"3306","short_description":"In a heavily polluted industrialized area in England about 87 percent of the…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['0.80, 0.48 or 48%', '0.55, 0.75 or 75%', '0.50, 0.50 or 50%', '0.60, 0.70 or 70%', '0.64, 0.53 or 53%', '0.77, 0.30 or 30%', '0.70, 0.40 or 40%', '0.87, 0.25 or 25%', '0.75, 0.60 or 60%', '0.47, 0.65 or 65%']\",\"question\":\"In a heavily polluted industrialized area in England about 87 percent of the moths of the speciesBistonbetulariaare melanic(dark colored). If the frequency of recessives is 0.13, what is the frequency of the dominant allele leading to melanism? What proportion of the dark colored moths are heterozygous?\",\"src\":\"stemez-Genetics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}