{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4483604b-4997-5c71-bfdd-cca3113486c8","task_key":"default--test--5662","task_revision_id":"1","upstream_id":"5662","short_description":"In what ways might international measures be employed? Are they a help or a…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Multi-partner initiatives to support the reforms necessary for the prevention of terrorism have proved ineffective. Sanctions against countries that provide a territorial base for terrorism may be misplaced, having negligible effects on the terrorist organizations themselves.', 'Countries often resist international measures out of concerns over sovereignty. This can limit the effectiveness of these measures and hinder international cooperation against terrorism.', 'A great deal of international diplomacy has involved attempts to define terrorism so that all countries could take steps to eradicate it. However, these efforts have largely faltered: countries often support or sympathize with dissidents who use violence against repressive governments; governments in developing countries have wanted to avoid anti-colonial struggles being labelled as terrorism; and countries want to retain a degree of flexibility regarding extradition and punishment of political dissidents.', 'Intelligence agencies are highly centralized, organized structures that operate at a maximum functional level within their domestic context. There is no reason to presume that collaboration would enhance their ability to contribute to the prevention of terrorism. The multi-agency approach would also broaden the scope for potential errors, miscommunication, and intelligence failures.']\",\"question\":\"In what ways might international measures be employed? Are they a help or a hindrance?\",\"src\":\"ori_mmlu-security_studies\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}