{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"555e376f-3ffb-5933-8d1e-22f9651aaa05","task_key":"default--test--2885","task_revision_id":"1","upstream_id":"2885","short_description":"In an isolated mountain village, the gene frequencies of A, B and O blood…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['O = 2, A = 387, B = 15, AB = 20', 'O = 1, A = 391, B = 0, AB = 32', 'O = 0, A = 392, B = 1, AB = 31', 'O = 7, A = 382, B = 20, AB = 15', 'O = 6, A = 388, B = 18, AB = 12', 'O = 4, A = 380, B = 17, AB = 23', 'O = 3, A = 390, B = 10, AB = 21', 'O = 0, A = 391, B = 1, AB = 32', 'O = 5, A = 385, B = 16, AB = 18', 'O = 0, A = 389, B = 2, AB = 33']\",\"question\":\"In an isolated mountain village, the gene frequencies of A, B and O blood alleles are 0.95, 0.04, and 0.01, respectively. If the total population is 424 calculate the number of individuals with O, A, B, and AB type blood.\",\"src\":\"stemez-Biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}