{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e517bd8e-a3db-5c36-91fb-8391ff0e98d5","task_key":"default--test--5464","task_revision_id":"1","upstream_id":"5464","short_description":"Brown cosigned Royal's $50000 note to State Bank. If Royal is later adjudicated…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Liable to pay State only if State first seeks payment from Royal.', 'Not liable to pay State as Brown was only a voluntary surety.', 'Liable to pay State only if Royal fails to meet the payment deadline.', \\\"Not liable to pay State, as Royal's mental incompetency releases all cosigners from liability.\\\", 'Not liable to pay State unless Brown was a compensated surety.', \\\"Liable to pay State, but only half of the note due to Royal's mental incompetency.\\\", \\\"Not liable to pay State because Royal's incompetency discharges Royal as a surety.\\\", 'Liable to pay State on the due date of the note.', \\\"Not liable to pay State, as Brown's liability is dependent on Royal's mental competency at the time of signing.\\\", \\\"Liable to pay State, but only if Royal's assets are insufficient to cover the debt.\\\"]\",\"question\":\"Brown cosigned Royal's $50000 note to State Bank. If Royal is later adjudicated mentally incompetent what would be Brown's liability on the note?\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}