{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"aa8d762c-13a4-5d1c-b768-8e9426c42ffa","task_key":"default--test--7156","task_revision_id":"1","upstream_id":"7156","short_description":"When the value of the U.S. dollar appreciates relative to other currencies which…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['More foreign visitors will travel to the United States.', 'Exports from the United States will increase.', 'U.S. residents will take more vacations in foreign countries.', 'Imports into the United States will decrease.', 'The value of other currencies will appreciate relative to the U.S. dollar.', 'The value of the U.S. dollar will depreciate relative to other currencies.', 'U.S. residents will take fewer vacations in foreign countries.']\",\"question\":\"When the value of the U.S. dollar appreciates relative to other currencies which of the following is the most likely to occur?\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}