{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bfa3ab0b-2177-5f6d-84ea-faf9e0d9281e","task_key":"default--test--6883","task_revision_id":"1","upstream_id":"6883","short_description":"In corporation XY sales in year 197X amount to $3,000,000; the cost of goods…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['$58,000 and $68,000', '$55,000 and $66,000', '$56,000 and $65,000', '$52,000 and $62,000', '$60,000 and $70,000', '$59,000 and $69,000', '$53,000 and $63,000', '$50,000 and $60,000', '$54,000 and $64,000', '$57,000 and $67,000']\",\"question\":\"In corporation XY sales in year 197X amount to $3,000,000; the cost of goods sold is $1,500,000; administrative ex-penses are $300,000; depreciation $800,000; all other ex-penses are $100,000; retained earnings are $100,000; when the corporation income tax rate is 48%, how much dividend is declared? How much could be declared if the tax rate fell to 45%?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}