{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8d1738f9-6c50-5e04-8eb8-c04b3b9ed2db","task_key":"default--test--1857","task_revision_id":"1","upstream_id":"1857","short_description":"A plaintiff sued an insurance company to recover life insurance proceeds. At…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"A diary entry from the insured's mother noting the day of the insured's birth.\\\", \\\"A family bible with the insured's birth date inscribed on the cover.\\\", 'A social media post from the insured, celebrating their birthday.', \\\"The insured's hospital birth record.\\\", \\\"A copy of the insured's driver's license with the birth date.\\\", \\\"A photocopy of the insured's birth certificate.\\\", 'A family portrait engraved with the ages of the family members.', \\\"A letter from the insured's childhood friend, recalling their shared birthdays.\\\", \\\"A school record indicating the insured's date of birth.\\\", \\\"A sworn affidavit from the insured's brother verifying the insured's birth date.\\\"]\",\"question\":\"A plaintiff sued an insurance company to recover life insurance proceeds. At trial, the issue before the court is the insured's correct date of birth. Which of the following is inadmissible if offered to prove the insured's correct date of birth?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}