{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"173d32df-3f28-5fa2-b742-424183d54712","task_key":"all--test--173d32df-3f28-5fa2-b742-424183d54712","task_revision_id":"1","upstream_id":"","short_description":"The distribution of salaries of a county school system with 4,752 employees is…","config":"all","split":"test","body":"{\"choices\":\"['the sample size is too small compared to the number of employees in the system to give a fair representation', 'the skewed salary distribution tells us that assumption of normality of the sampled population will not be satisfied', 'the population standard deviation of the salaries of employees of this county school system is not known', 'the teacher salaries are not negotiable, but the superintendent’s salary is negotiable, so the superintendent’s salary should be excluded from the population sampled']\",\"question\":\"The distribution of salaries of a county school system with 4,752 employees is known to be right skewed, with the superintendent’s salary an outlier on the higher side. A random sample of 20 employees was taken and their salaries recorded. A 95 percent t-confidence interval for the mean salary of the county school system employees is ($15,360, $32,470). The t-confidence interval is not appropriate in this situation because\",\"subject\":\"high_school_statistics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}