{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"124c78fd-1724-5220-93f0-8c2608273486","task_key":"all--test--124c78fd-1724-5220-93f0-8c2608273486","task_revision_id":"1","upstream_id":"","short_description":"A heavy rock and a light rock in free fall (zero air resistance) have the same…","config":"all","split":"test","body":"{\"choices\":\"['force due to gravity is the same on each.', 'air resistance is always zero in free fall.', 'inertia of both rocks is the same.', 'ratio of force to mass is the same.']\",\"question\":\"A heavy rock and a light rock in free fall (zero air resistance) have the same acceleration. The heavy rock doesn't have a greater acceleration because the\",\"subject\":\"conceptual_physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}