{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"14a8dd78-7023-586f-a623-5f372b658e33","task_key":"default--test--5365","task_revision_id":"1","upstream_id":"5365","short_description":"Considering 'offensive' and 'defensive realism', which of these statements is…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Both offensive and defensive realism agree that states should avoid cooperation to achieve security.', 'Offensive realism regards cooperation as the way in which states can achieve security.', 'Offensive realism argues that states should avoid power and competition to achieve security.', 'Both defensive realism and offensive realism regard power and competition as a means by which security can be achieved for the state.', 'Defensive realism theorizes that states will pursue hegemony where possible.', 'Both offensive and defensive realism agree that states should avoid power and competition to achieve security.']\",\"question\":\"Considering 'offensive' and 'defensive realism', which of these statements is not correct?\",\"src\":\"ori_mmlu-security_studies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}