{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1d026e42-dedd-592f-ac21-86cccdd4e6be","task_key":"default--test--7497","task_revision_id":"1","upstream_id":"7497","short_description":"Classical economists generally believe that I. wages fluctuate quickly II. Say's…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['II, III and IV only', 'II and III only', 'I and II only', 'I and IV only', 'I and III only', 'I III and IV only', 'II and IV only', 'I, II and III only', 'III and IV only']\",\"question\":\"Classical economists generally believe that I.   wages fluctuate quickly II.   Say's law does not hold III.   input and output prices will stay in line with each other IV.   the government should not worry about maintaining aggregate demand at an adequate level\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}