{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7f94719a-8f0e-5f98-94ce-040332e35cdb","task_key":"all--test--7f94719a-8f0e-5f98-94ce-040332e35cdb","task_revision_id":"1","upstream_id":"","short_description":"Ms. Ling has been living in her home for exactly 4 years. She knows that there…","config":"all","split":"test","body":"{\"choices\":\"['12 ÷ m = 4', 'm × 4 = 12', '12 – m = 4', 'm ÷ 4 = 12']\",\"question\":\"Ms. Ling has been living in her home for exactly 4 years. She knows that there are 12 months in each year. She found the total number of months, m, she has been living in her home by using the number sentence below. 4 × 12 = m Which number sentence shows another way to find the total number of months, m, Ms. Ling has been living in her home?\",\"subject\":\"elementary_mathematics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}