{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"929941b9-de97-5853-a00b-b734fb4bbf67","task_key":"default--test--5465","task_revision_id":"1","upstream_id":"5465","short_description":"A taxpayer purchases and is the owner of an insurance contract on his own life…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Two.', 'Five.', 'Zero.', 'One.', 'A gift is made each time a premium payment is made.', 'Six.', 'Two for each premium payment.', 'Three.', 'One for each premium payment.', 'Four.']\",\"question\":\"A taxpayer purchases and is the owner of an insurance contract on his own life and designates his two children as equal beneficiaries. The taxpayer makes all premium payments. How many gifts of property, if any, have been made for gift tax purposes?\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}