{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"21168fa7-9f96-5d61-8bbb-8db126891704","task_key":"default--test--3299","task_revision_id":"1","upstream_id":"3299","short_description":"Kernel color in wheat is determined by the action of two pairs ofpolygenesthat…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['(a) 3/4, (b) Two', '(a) 1/16, (b) Nine', '(a) 3/16, (b) Six', '(a) 6/16, (b) Four', '(a) 5/16, (b) Seven', '(a) 4/16, (b) Three', '(a) 2/16, (b) Five', '(a) 1/4, (b) Seven', '(a) 1/16, (b) Four', '(a) 1/8, (b) Eight']\",\"question\":\"Kernel color in wheat is determined by the action of two pairs ofpolygenesthat produce colors varying from dark red to white. If AABB (dark red) andaabb(white) are crossed, (a) What fraction of the F_2 generation can be expected to be like either parent? (b) How many F_2 phenotypic classes result?\",\"src\":\"stemez-Genetics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}