{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d88b3dcd-69ca-5f52-9489-7c125c37b796","task_key":"default--test--1957","task_revision_id":"1","upstream_id":"1957","short_description":"A sister and brother own a ranch as joint tenants. If the sister transfers her…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['A vested remainder interest in the property.', 'A life estate interest in the ranch.', 'Full ownership of the property.', 'An undivided one-half interest with right of survivorship.', 'An undivided one-half interest without right of survivorship.', 'A lien against the entire property.', 'A tenancy in common with the brother.', 'A claim to future profits from the ranch.', 'A leasehold interest in the property.']\",\"question\":\"A sister and brother own a ranch as joint tenants. If the sister transfers her interest by quitclaim deed to her friend without the brother's knowledge, what interest, if any, does the friend have in the ranch?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}