{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fb933d8d-e93e-5697-b667-e5c1c0b1e2be","task_key":"default--test--7152","task_revision_id":"1","upstream_id":"7152","short_description":"A stronger stock market is likely to cause which of the following changes in the…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['No change \\\\xa0\\\\xa0\\\\xa0 Decrease', 'Increase \\\\xa0\\\\xa0\\\\xa0 Increase', 'Decrease \\\\xa0\\\\xa0\\\\xa0 Increase', 'No change \\\\xa0\\\\xa0\\\\xa0 Increase', 'Increase \\\\xa0\\\\xa0\\\\xa0 No change', 'Increase \\\\xa0\\\\xa0\\\\xa0 Decrease', 'No change \\\\xa0\\\\xa0\\\\xa0 No change', 'Decrease \\\\xa0\\\\xa0\\\\xa0 No change', 'Decrease \\\\xa0\\\\xa0\\\\xa0 Decrease']\",\"question\":\"A stronger stock market is likely to cause which of the following changes in the consumption function and aggregate demand? CONSUMPTION FUNCTION     AGGREGATE DEMAND\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}