{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"568f8715-ef72-5206-aba3-05cbe9615693","task_key":"default--test--8868","task_revision_id":"1","upstream_id":"8868","short_description":"A doctor is concerned about the relationship between blood pressure and…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['18.7%', '10.2%', '12.5%', '14.0%', '8.6%', '22.9%', '20.4%', '15.1% ', '17.3%', '13.8%']\",\"question\":\"A doctor is concerned about the relationship between blood pressure and irregular heartbeats. Among her patients, she classifies blood pressures as high, normal, or low and heartbeats as regular or irregular and finds that 16\\\\% have high blood pressure; (b) 19\\\\% have low blood pressure; (c) $17 \\\\%$ have an irregular heartbeat; (d) of those with an irregular heartbeat, $35 \\\\%$ have high blood pressure; and (e) of those with normal blood pressure, $11 \\\\%$ have an irregular heartbeat. What percentage of her patients have a regular heartbeat and low blood pressure?\",\"src\":\"scibench-stat\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}