{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"735df02c-e1c0-5e40-9a83-11e210550344","task_key":"default--test--6978","task_revision_id":"1","upstream_id":"6978","short_description":"Which of the following is not among the methods unions use to increase wages?","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"[\\\"Seeking to increase the demand for the union's labor\\\", 'Negotiations to obtain a wage floor', 'Encouraging workers to increase their productivity', 'Lobbying for higher minimum wage laws', 'Promoting worker training and education', 'Restrictive membership policies', 'Efforts to decrease the prices of substitute resources', 'Featherbedding or make-work rules', 'Encouraging government regulation of industries', 'Instituting strike actions']\",\"question\":\"Which of the following is not among the methods unions use to increase wages?\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}