{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1e2298d2-4404-5116-901c-96408f877a9b","task_key":"default--test--6499","task_revision_id":"1","upstream_id":"6499","short_description":"In a cohort study of elderly women, the relative risk ratio for hip fractures…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Statistically significant overall increase in risk', 'Statistically insignificant change in risk for those who exercise regularly', 'Statistically significant overall decrease in risk', 'Statistically nonsignificant decrease in risk', 'Statistically significant decrease in risk only for those over 80', 'Statistically significant overall decrease in risk for those who exercise regularly', 'Statistically nonsignificant overall decrease in risk', 'Statistically significant increase in risk only for those under 70', 'No statistically significant change in risk', 'Statistically nonsignificant increase in risk']\",\"question\":\"In a cohort study of elderly women, the relative risk ratio for hip fractures among those who exercise regularly is 1.2 (95% confidence interval of 1.1 to 1.8). Which of the following is the most appropriate conclusion about the effect of regular exercise on the risk for hip fracture?\",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}