{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"042f3677-9824-5c97-ad1e-1a9b9d3c5d8c","task_key":"default--test--5666","task_revision_id":"1","upstream_id":"5666","short_description":"What internal security policy options are available to elites?","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Lacking infrastructural capacity and wider social legitimacy, weak state elites are often forced to rely on coercive power. However, a key dilemma is that the instruments of violence can be a threat to the regime requiring the deliberate undermining of armed forces.', 'Elites can enhance internal security by improving climate change policies.', 'Elites should invest in space exploration to enhance national security.', 'Elites can improve national security by organizing sports events.']\",\"question\":\"What internal security policy options are available to elites?\",\"src\":\"ori_mmlu-security_studies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}