{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"96f2295e-fea8-53aa-86a7-498bb7667d49","task_key":"default--test--7351","task_revision_id":"1","upstream_id":"7351","short_description":"All else equal when the United States exports more goods and services","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['the value of the dollar rises as the supply of dollars decreases.', 'the value of the dollar falls as demand for dollars decreases.', 'the value of the dollar falls as the supply of dollars decreases.', 'the value of the dollar remains the same regardless of the supply and demand for dollars.', 'the value of the dollar falls as the supply of dollars increases.', 'the value of the dollar rises as demand for dollars increases.', 'the value of the dollar remains stable as the supply and demand for dollars are balanced.', 'the value of the dollar rises as the supply of dollars increases.', 'the value of the dollar falls, as the demand for dollars is lower than the supply.', 'the value of the dollar rises, despite a decrease in demand for dollars.']\",\"question\":\"All else equal when the United States exports more goods and services\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}