{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2435f263-6f4d-5023-9dea-afa6cce66ce6","task_key":"default--test--7912","task_revision_id":"1","upstream_id":"7912","short_description":"A teacher believes that giving her students a practice quiz every week will…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"[\\\"An observational study, because the teacher merely observed the students' progress\\\", 'An observational study, because there was no use of a control group', 'An observational study, but a poorly designed one because randomization was not used', 'An experiment, thus making cause and effect a reasonable conclusion', 'An observational study, but the results cannot be generalized to other classes', 'An observational study, but a well-designed one because all students were observed', 'An experiment, but the outcome cannot definitively be attributed to the quizzes', 'An experiment, but a poorly designed one because there was no control group', 'An experiment, but with no reasonable conclusion possible about cause and effect', 'An experiment, because the teacher was actively manipulating the environment']\",\"question\":\"A teacher believes that giving her students a practice quiz every week will motivate them to study harder, leading to a greater overall understanding of the course material. She tries this technique for a year, and everyone in the class achieves a grade of at least C. Is this an experiment or an observational study?\",\"src\":\"ori_mmlu-high_school_statistics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}