{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ffa43877-eaf9-5ec6-a1f1-d25cda520b3a","task_key":"all--test--ffa43877-eaf9-5ec6-a1f1-d25cda520b3a","task_revision_id":"1","upstream_id":"","short_description":"A plaintiff sued an insurance company to recover life insurance proceeds. At…","config":"all","split":"test","body":"{\"choices\":\"['A family portrait engraved with the ages of the family members.', \\\"A family bible with the insured's birth date inscribed on the cover.\\\", \\\"A photocopy of the insured's birth certificate.\\\", \\\"A sworn affidavit from the insured's brother verifying the insured's birth date.\\\"]\",\"question\":\"A plaintiff sued an insurance company to recover life insurance proceeds. At trial, the issue before the court is the insured's correct date of birth. Which of the following is inadmissible if offered to prove the insured's correct date of birth?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}