{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"40882607-0794-5daf-8765-58a9198d4790","task_key":"default--test--1360","task_revision_id":"1","upstream_id":"1360","short_description":"A wife and husband are married and own a dairy as tenants by the entirety. If…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['A temporary leasehold interest.', 'An undivided one-half interest with right of survivorship.', 'An undivided one-third interest with right of survivorship.', 'The right to inhabit the dairy.', 'An undivided one-half interest without right of survivorship.', 'A life estate in the dairy.', 'A legal claim to future profits from the dairy.', 'No interest.', 'A lien against the entire property.', 'Full ownership of the property.']\",\"question\":\"A wife and husband are married and own a dairy as tenants by the entirety. If the wife transfers her interest in the dairy by quitclaim deed to her aunt without the husband's knowledge, what interest, if any, does the aunt have?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}