{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6c2e889a-abbe-515e-aeaf-a35c32900c9b","task_key":"default--test--8648","task_revision_id":"1","upstream_id":"8648","short_description":"5.4-17. In a study concerning a new treatment of a certain disease, two groups…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['$0.1745$', '$0.3751$', '$0.5012$', '$0.3359$', '$0.3926$', '$0.2872$', '$0.6118$', '$0.4256$', '$0.4583$', '$0.2197$']\",\"question\":\"5.4-17. In a study concerning a new treatment of a certain disease, two groups of 25 participants in each were followed for five years. Those in one group took the old treatment and those in the other took the new treatment. The theoretical dropout rate for an individual was $50 \\\\%$ in both groups over that 5 -year period. Let $X$ be the number that dropped out in the first group and $Y$ the number in the second group. Assuming independence where needed, give the sum that equals the probability that $Y \\\\geq X+2$. HINT: What is the distribution of $Y-X+25$ ?\\n\",\"src\":\"scibench-stat\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}