{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a835539f-5d6f-5664-9cb5-79a82ef75a8b","task_key":"default--test--825","task_revision_id":"1","upstream_id":"825","short_description":"The owners of independent gasoline stations in a community meet secretly and…","config":"default","split":"test","body":"{\"category\":\"business\",\"options\":\"['found guilty under the Sherman Antitrust Act and face prison time', 'found guilty under the Robinson-Patman Act and face prison time', 'found guilty under the Magnuson-Moss Warranty Act and face large fines', 'found not guilty because they did not engage in any form of price fixing', 'found guilty under the Adamson Act and face prison time and fines', 'found guilty under the Clayton Act and face both prison time and fines', 'found not guilty because there is no law against price collusion in this industry', 'found guilty under the Dodd-Frank Wall Street Reform and Consumer Protection Act and face fines', 'found not guilty because retailers are allowed to set their own price levels', 'found guilty under the Federal Trade Commission Act and face fines']\",\"question\":\"The owners of independent gasoline stations in a community meet secretly and agree to keep their prices high to protect their profitability. If their plan is discovered and they are charged with a crime, they could be\",\"src\":\"ori_mmlu-marketing\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}