{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7c57e695-a0cc-52c1-b602-b2a939bfb67f","task_key":"default--test--5077","task_revision_id":"1","upstream_id":"5077","short_description":"Which of the following is an important reason to advocate the process of…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['To prevent the erosion of established civil liberties through the removal of special rights and the cessation of extraordinary measures.', 'To prevent stagnation of the security realm and facilitate the recognition and securitization of new or more urgent existential threats.', 'To ensure that securitizing actors conceptualise security threats in terms of those that can be addressed using extraordinary measures.']\",\"question\":\"Which of the following is an important reason to advocate the process of desecuritization?\",\"src\":\"ori_mmlu-security_studies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}