{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1eb4f7bc-e095-5274-98c9-e9dd333cfbf9","task_key":"default--test--7282","task_revision_id":"1","upstream_id":"7282","short_description":"What is the elasticity of demand?","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Elasticity of demand is the ratio of income change to the change in quantity demanded', 'Elasticity of demand is the total revenue divided by the change in quantity of a good sold', 'Elasticity of demand is the measure of the speed at which goods are produced in response to market demands', 'Elasticity of demand is the change in quantity demanded in response to changes in market conditions other than price', 'Elasticity of demand is the change in quantity supplied due to a change in consumer preferences', 'Elasticity of demand is a measure of the extent to which quantity of a good demanded responds to changes in the price of the good', 'Elasticity of demand is the ratio of price to quantity demanded', 'Elasticity of demand is the percentage change in price', 'Elasticity of demand is the change in quantity demanded', 'Elasticity of demand is the change in the price of one good in response to a change in the price of another good']\",\"question\":\"What is the elasticity of demand?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}