{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d03df8c1-6666-5a9d-8f9d-eca28643b3d6","task_key":"default--test--1484","task_revision_id":"1","upstream_id":"1484","short_description":"Donny threw a knife at Sally, intending to injure her severely. However, Donny…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Donny will be liable for battery, but not assault.', 'Donny will not be liable for either assault or battery because he did not actually hit Sally with the knife.', 'Donny will be liable for assault, but not battery because he missed Sally with the knife.', 'Donny will be liable for assault and for battery.', 'Donny will not be liable for either assault or battery because this is only a criminal matter.', 'Donny will not be liable for either assault or battery because Sally was not physically harmed.', 'Donny will be liable for assault and battery because he intended to cause harm.', 'Donny will be liable for both assault and battery because Sally was terrified.', 'Donny will be liable for battery, but not assault because Sally was able to dodge the knife.', 'Donny will be liable for assault, but not battery.']\",\"question\":\"Donny threw a knife at Sally, intending to injure her severely. However, Donny missed Sally. Sally saw the knife just as it whizzed by her head, missing it by about one inch. As a result, Sally was very scared. Sally sued Donny for assault and battery. Which of the following is most correct?\",\"src\":\"ori_mmlu-jurisprudence\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}