{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"320617e6-3d83-52b6-a13e-a0d2e8afaf10","task_key":"default--test--1147","task_revision_id":"1","upstream_id":"1147","short_description":"What is it called when a remainder in the grantor's heirs is invalid and becomes…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Related Doctrine of Merger', 'Contingent remainder', 'Doctrine of Worthier Title', 'Doctrine of Escheat', 'Vested remainder', \\\"Rule in Wild's Case\\\", 'Doctrine of Lapse', \\\"Rule in Shelley's Case\\\", 'Doctrine of Advowson', 'Rule Against Perpetuities']\",\"question\":\"What is it called when a remainder in the grantor's heirs is invalid and becomes a reversion in the grantor?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}