{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ff056cd1-3d30-507f-b011-bd3ab9664843","task_key":"default--test--3399","task_revision_id":"1","upstream_id":"3399","short_description":"One pair of genes for coat color in cats is sex-linked. The gene B produces…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['(1/4) tortoise-shell females, (1/4) black females, (1/4) yellow males, and (1/4) black males', 'All yellow males', '(1/2) yellow females and (1/2) black males', '(1/2) black females and (1/2) yellow males', '(1/3) black males, (1/3) yellow males, and (1/3) tortoiseshell females', 'All black offspring', 'All black males', '(1/2) yellow males and (1/2) tortoiseshell females', 'All tortoise-shell females', '(1/2) tortoiseshell females and (1/2) black males']\",\"question\":\"One pair of genes for coat color in cats is sex-linked. The gene B produces yellow coat, b produces black coat, and the heterozygous Bb produces tortoiseshell coat. What kind of offspring will result from the mating of a black male and a tortoiseshell female?\",\"src\":\"stemez-Genetics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}