{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"baa4b08a-b737-5fe2-bf39-1977976a1780","task_key":"all--test--baa4b08a-b737-5fe2-bf39-1977976a1780","task_revision_id":"1","upstream_id":"","short_description":"The first number in a number pattern is 28. The pattern rule is to add 14 to get…","config":"all","split":"test","body":"{\"choices\":\"['All the numbers in the pattern can be divided equally by 10.', 'All the numbers in the pattern can be divided equally by 4.', 'All the numbers in the pattern can be divided equally by 8.', 'All the numbers in the pattern can be divided equally by 7.']\",\"question\":\"The first number in a number pattern is 28. The pattern rule is to add 14 to get the next number in the pattern. If the pattern continues, which statement is true?\",\"subject\":\"elementary_mathematics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}