{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"57f92c1d-70d6-5c60-a813-b837b8c6cb42","task_key":"default--test--7151","task_revision_id":"1","upstream_id":"7151","short_description":"Of the following choices the most direct exchange in the circular flow model of…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['households provide revenues to firms in exchange for resources.', 'firms supply goods to households in exchange for goods.', 'households provide wage payments to firms in exchange for resources.', 'households provide goods to firms in exchange for resources.', 'firms supply wage payments to households in exchange for revenues.', 'households provide revenues to firms in exchange for wage payments.', 'households provide goods to firms in exchange for wage payments.', 'firms supply goods to households in exchange for revenues.', 'households provide resources to firms in exchange for goods.', 'firms supply resources to households in exchange for wage payments.']\",\"question\":\"Of the following choices the most direct exchange in the circular flow model of a private closed economy is when\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}