{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3203b93b-5b21-52fd-acb5-23694eb66e38","task_key":"default--test--976","task_revision_id":"1","upstream_id":"976","short_description":"What is personal (ratione personae) immunity?","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Personal immunity is given to individuals for their private conduct, not their public conduct', 'Personal immunity is afforded to all physical persons', 'Personal immunity is afforded to all State officials, regardless of their conduct', 'Personal immunity is only given to individuals who have committed a crime in a public capacity', 'Personal immunity is afforded only to particular persons irrespective if their conduct was undertaken in a private or public capacity', 'Personal immunity is given to all individuals, regardless of their conduct', 'Personal immunity is that which is afforded in a personal capacity and hence does not cover conduct of the State as such', 'Personal immunity is afforded to State officials for conduct undertaken in a public capacity', 'Personal immunity is afforded to State officials for conduct undertaken in a private capacity.', 'Personal immunity is afforded only to particular persons if their conduct was undertaken in a public capacity']\",\"question\":\"What is personal (ratione personae) immunity?\",\"src\":\"ori_mmlu-international_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}