{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"de956810-821c-5395-a607-3ec00600f8d3","task_key":"default--test--7223","task_revision_id":"1","upstream_id":"7223","short_description":"An American buys an entertainment system that was manufactured in China. How…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Net exports and GDP both fall.', 'GDP falls and there is no change in net exports.', 'Net exports rise and there is no change in GDP.', 'Net exports and GDP both rise.', 'Net exports rise and GDP falls.', 'There is no change in either net exports or GDP.', 'Net exports and GDP go in opposite directions.', 'Net exports fall and GDP rises.', 'GDP rises and there is no change in net exports.', 'Net exports fall and there is no change in GDP.']\",\"question\":\"An American buys an entertainment system that was manufactured in China. How does the U.S. national income accounts treat this transaction?\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}