{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d8fae2a9-f22d-5551-97b1-44e12ccdbf92","task_key":"default--test--7600","task_revision_id":"1","upstream_id":"7600","short_description":"Differentiate the national security argument from the cheap foreign labor…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['National security argument is about controlling immigration, while cheap foreign labor argument is about promoting free trade.', 'National security argument is about protecting consumers, while cheap foreign labor argument is about encouraging domestic innovation.', 'National security argument is about protecting essential industries for defense, while cheap foreign labor argument is about protecting domestic jobs and wages.', 'National security argument is about economic growth, while cheap foreign labor argument is about protecting domestic industries.', 'National security argument is about enhancing cultural identity, while cheap foreign labor argument is about preserving traditional industries.', 'National security argument is about environmental protection, while cheap foreign labor argument is about ensuring ethical labor practices abroad.', 'National security argument is about maintaining political stability, while cheap foreign labor argument is about improving living standards.', 'National security argument is about avoiding dependence on foreign technology, while cheap foreign labor argument is about maintaining competitive tax rates.', 'National security argument is about military power, while cheap foreign labor argument is about reducing production costs.', 'National security argument is about safeguarding national sovereignty, while cheap foreign labor argument is about preventing wage inflation.']\",\"question\":\"Differentiate the national security argument from the cheap foreign labor argument in favor of protection.\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}