{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"893ad0ee-8c95-5b04-b9d6-bd851e6cdc1d","task_key":"default--test--7535","task_revision_id":"1","upstream_id":"7535","short_description":"What might happen to the housing construction industry if com-mercial banks are…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['The quality of housing construction would deteriorate', 'More houses could be bought', 'The housing industry would slump', 'Mutual savings banks would become obsolete', 'Deposits in mutual savings banks would increase', 'Housing prices would become more volatile', 'The housing industry would boom', 'Commercial banks would solely focus on large-scale developments', 'There would be no noticeable change in the housing construction industry']\",\"question\":\"What might happen to the housing construction industry if com-mercial banks are allowed to compete freely with mutual sav-ings banks?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}