{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"45780c5f-1890-5d64-b725-8ebece9b9631","task_key":"default--test--7169","task_revision_id":"1","upstream_id":"7169","short_description":"Which function of money best defines $1.25 as the price of a 20-ounce bottle of…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Store of value', 'Medium of exchange', 'Unit of account', 'Measure of liquidity', 'Measure of credit', 'Indicator of wealth', 'Standard for deferred payment', 'Mode of transfer', 'Transfer of ownership', 'Benchmark of exchange rate']\",\"question\":\"Which function of money best defines $1.25 as the price of a 20-ounce bottle of pop?\",\"src\":\"ori_mmlu-high_school_macroeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}