{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1477c2ca-f878-5457-9c2b-41c0a5c43931","task_key":"default--test--1125","task_revision_id":"1","upstream_id":"1125","short_description":"What is an act jure gestionis?","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Acts jure gestionis are military actions undertaken by States', 'All diplomatic missions undertaken by the State are acts jure gestionis', 'All international treaties signed by the State are acts jure gestionis', 'All purchases by the State are acts jure gestionis', 'All acts undertaken by State corporations are jure gestionis', 'Acts jure gestionis are all legislative decisions made by the State.', 'Acts jure gestionis are those undertaken by States in a private capacity', 'All conduct undertaken by government officials in their free time is considered jus gestionis.', 'Acts jure gestionis are all actions performed by the State in a public capacity', 'Acts jure gestionis are all decisions made by State courts']\",\"question\":\"What is an act jure gestionis? \",\"src\":\"ori_mmlu-international_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}