{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4a27e706-a613-514c-9ed2-bba9810e44f2","task_key":"default--test--7260","task_revision_id":"1","upstream_id":"7260","short_description":"Indicate whether the following economic events represent a change in supply or a…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['a) None of the above, b) Change in supply, c) Change in quantity supplied', 'a) Change in quantity supplied, b) Change in quantity supplied, c) Change in supply', 'a) None of the above, b) None of the above, c) None of the above', 'a) Change in supply, b) Change in quantity supplied, c) None of the above', 'a) Change in quantity supplied, b) None of the above, c) Change in supply', 'a) Change in supply, b) Change in quantity supplied, c) Change in supply', 'a) Change in supply, b) Change in supply, c) Change in quantity supplied', 'a) Change in quantity supplied, b) Change in supply, c) Change in supply', 'a) Change in supply, b) Change in quantity supplied, c) Change in quantity supplied', 'a) Change in quantity supplied, b) Change in supply, c) Change in quantity supplied']\",\"question\":\"Indicate whether the following economic events represent a change in supply or a change in quantity supplied. a) Cold weather in Florida destroys a great deal of the orange crop. Therefore there are fewer oranges in the country's supermarkets. b) The cost of limes increase. As a result, bar owners raise the price of gin and tonics. c) The price of leather shoes goes up. This induces manufacturers of leather shoes to increase production.\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}