{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"12acf287-3695-550b-b5fa-f9a1f5097058","task_key":"default--test--5774","task_revision_id":"1","upstream_id":"5774","short_description":"According to the SEC, members of an issuer’s audit committee may not","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Establish procedures for the receipt, retention, and treatment of complaints received by the registrant regarding accounting, internal accounting controls, or auditing matters.', 'Engage independent counsel as deemed necessary to carry out their duties.', 'Accept any consulting, advisory, or other compensatory fee from the registrant for services other than as a member of the board.', 'Serve on the board of directors of another company.', 'Establish procedures for employees to anonymously report fraud.', 'Serve on more than three audit committees of public companies.']\",\"question\":\"According to the SEC, members of an issuer’s audit committee may not\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}