{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f7b6be29-d522-57e0-9ce7-b4c104c6494e","task_key":"default--test--7175","task_revision_id":"1","upstream_id":"7175","short_description":"Consider a profit-maximizing firm in a perfectly competitive market with several…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"[\\\"lead the firm to reduce workers' wages as they now contribute less marginal revenue\\\", 'lead the firm to hire more workers but not to raise their wages', \\\"reduce the firm's employment level but raise workers' wages\\\", 'lead the firm to hire more workers and to pay them higher wages', 'raise the workers’ hourly wage as they now contribute more marginal revenue', \\\"reduce the firm's employment level and lower workers' wages\\\", 'cause the firm to hire fewer workers but pay them higher wages', 'lead the firm to hire the same number of workers but pay them higher wages', 'reduce the firm’s employment level, because fewer workers are now needed', \\\"have no effect on the firm's employment level or workers' wages\\\"]\",\"question\":\"Consider a profit-maximizing firm in a perfectly competitive market with several sellers and several buyers (i.e., the firm is a \\\"price taker\\\" of the goods it sells and a \\\"price taker\\\" of the hourly wages it pays its workers). If a technological innovation made by someone in this firm were to significantly raise the firm's marginal physical product (but not that of any other firm's), then this innovation would\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}