{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"42251d3d-19f2-58de-bdf5-2b21d266600c","task_key":"all--test--42251d3d-19f2-58de-bdf5-2b21d266600c","task_revision_id":"1","upstream_id":"","short_description":"An educational psychologist is administering a basic skills exam to…","config":"all","split":"test","body":"{\"choices\":\"['The psychologist is comparing two different schools.', 'The psychologist is comparing the same grade in each school.', 'The psychologist is testing the students in the two schools at two different times.', 'The psychologist is testing the students in the two schools on the same day.']\",\"question\":\"An educational psychologist is administering a basic skills exam to second-graders of two different schools in order to compare the students' performance. The researcher administers the exam to the students of the Antrim School on a Wednesday morning and then administers the same exam in exactly the same fashion on that same Wednesday afternoon to the second-graders of the Barton School. Which of the following best identifies a confounding variable in the psychologist's research?\",\"subject\":\"high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}