{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"734f51d7-139f-5e10-be69-bb18f6d37a09","task_key":"all--test--734f51d7-139f-5e10-be69-bb18f6d37a09","task_revision_id":"1","upstream_id":"","short_description":"A kidney dialysis center periodically checks a sample of its equipment and…","config":"all","split":"test","body":"{\"choices\":\"['Dialysis center: Type I error, towel manufacturer: Type I error', 'Dialysis center: Type I error, towel manufacturer: Type II error', 'Dialysis center: Type II error, towel manufacturer: Type I error', 'Dialysis center: Type II error, towel manufacturer: Type II error']\",\"question\":\"A kidney dialysis center periodically checks a sample of its equipment and performs a major recalibration if readings are sufficiently off target. Similarly, a fabric factory periodically checks the sizes of towels coming off an assembly line and halts production if measurements are sufficiently off target. In both situations, we have the null hypothesis that the equipment is performing satisfactorily. For each situation, which is the more serious concern, a Type I or Type II error?\",\"subject\":\"high_school_statistics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}