{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2befb854-504a-5a9a-956a-7438b8281504","task_key":"default--test--2401","task_revision_id":"1","upstream_id":"2401","short_description":"Professor Ek is interested in studying online bullying behaviors of middle…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Focus group - Discussing online bullying in a group setting would provide a range of perspectives and insights into the problem.', 'Naturalistic observation-observing instances of online bullying by monitoring middle school students\\\\' social networking behavior would provide the most accurate data and not raise important privacy concerns, because social networks are considered \\\"public\\\" space.', 'Content analysis - Analyzing the content of online messages and posts could provide insights into the nature and extent of online bullying.', 'Cross-sectional study - Comparing different groups of students at the same point in time would provide insights into how online bullying behavior varies across different ages, grades, or other demographic factors.', 'Survey-self-reports about online bullying behaviors and responses to these behaviors would likely provide the most accurate descriptions of bullying and its impact.', 'Case study - Studying a specific instance of online bullying in detail will provide the most comprehensive understanding of the behavior.', 'Ethnography - Immersing himself in the online environments of the students would allow Professor Ek to understand the cultural context of online bullying.', 'Longitudinal study - Tracking the same group of students over several years would allow Professor Ek to see patterns and changes in online bullying behavior over time.', 'Correlation-it would be unethical to purposefully expose middle school students to bullying behaviors, so Professor Ek should examine variables that correlate with existing instances of online bullying.', 'Experiment-the most convenient and ethical way to study online bullying is to randomly assign half of the participants to a group who experiences the independent variable (online bullying).']\",\"question\":\"Professor Ek is interested in studying online bullying behaviors of middle school students. Which of the following research method choices (and the rationale for the choice) is the most appropriate?\",\"src\":\"ori_mmlu-high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}