{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ba8e2ec1-227c-551e-a10c-e6cdf92d9e4a","task_key":"default--test--7597","task_revision_id":"1","upstream_id":"7597","short_description":"Suppose as a result of certain economic developments the priceof cars and the…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Increased taxes on automobile sales', 'Decreased demand for cars', 'Increased demand for cars', 'Decreased supply of steel', 'Technological advancements in car manufacturing']\",\"question\":\"Suppose as a result of certain economic developments the priceof cars and the price of steel both increase. What might bethe reasons behind this upsurge in prices, assuming the automobileindustry is fairly competitive and the supply of steelis constant in the short run?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}