{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a3aeb64b-01bd-5483-a6b1-e71d96862888","task_key":"default--test--8336","task_revision_id":"1","upstream_id":"8336","short_description":"A medical research team tests for tumor reduction in a sample of patients using…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['There are three levels of a single explanatory variable.', 'There are three explanatory variables and one response variable.', 'There is one explanatory variable with three levels of response.', 'There is one explanatory variable and three corresponding response variables.', 'There are three separate explanatory variables, each with a single level of response.', 'There are three response variables and one explanatory variable.', 'Tumor reduction is the only response variable, but there are three explanatory variables corresponding to the different dosages.', 'Tumor reduction is the only explanatory variable, but there are three response variables corresponding to the different dosages.', 'There are three response variables, each corresponding to a different level of a single explanatory variable.', 'There is one response variable with three levels of explanation.']\",\"question\":\"A medical research team tests for tumor reduction in a sample of patients using three different dosages of an experimental cancer drug. Which of the following is true?\",\"src\":\"ori_mmlu-high_school_statistics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}