{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e6d36ffd-ba2f-5ff6-acb0-e581ae888c8b","task_key":"default--test--8442","task_revision_id":"1","upstream_id":"8442","short_description":"A reading specialist in a large public school system believes that the more time…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['Yes, because the control group is necessary for comparison.', 'Yes.', 'No, because without blinding, there is a strong chance of a placebo effect.', 'No, because grade level is a lurking variable which may well be confounded with the variables under consideration.', 'No, because while this design may point out an association between reading and GPA, it cannot establish a cause-and-effect relationship.', 'No, because the experiment does not account for the varying reading levels of the students.', 'Yes, because the independent variable is manipulated by the researcher.', 'No, because the experimental design does not include a double-blind procedure.', 'Yes, because it tests the effect of different amounts of reading on GPA.', 'No, because the sample size of each group is not large enough.']\",\"question\":\"A reading specialist in a large public school system believes that the more time students spend reading, the better they will do in school. She plans a middle school experiment in which an SRS of 30 eighth graders will be assigned four extra hours of reading per week, an SRS of 30 seventh graders will be assigned two extra hours of reading per week, and an SRS of 30 sixth graders with no extra assigned reading will be a control group. After one school year, the mean GPAs from each group will be compared. Is this a good experimental design?\",\"src\":\"ori_mmlu-high_school_statistics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}