{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d2f006fb-0432-5b81-a478-bc17c652d8d9","task_key":"default--test--2617","task_revision_id":"1","upstream_id":"2617","short_description":"Statistical regression is a potential threat to a study’s internal validity…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['the researcher is unable to randomly assign participants to the different treatment groups.', \\\"there is a statistically significant difference in the gender distribution among the study's treatment groups.\\\", 'participants are selected for inclusion in the study because of their extremely high or low scores on the pretest.', 'participants are selected for inclusion in the study based on their availability.', 'there is a statistically different number of participants in the study’s treatment groups.', 'the researcher has a personal bias towards a certain result.', 'the study is conducted in a controlled laboratory setting.', 'there is a statistically significant difference in the number of participants who drop out of the different treatment groups during the course of the study.', 'the researcher does not use a control group in the study.', \\\"the study's data is not normally distributed.\\\"]\",\"question\":\"Statistical regression is a potential threat to a study’s internal validity when:\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}