{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"acafa190-277f-586f-a427-e0a044a59b98","task_key":"default--test--7239","task_revision_id":"1","upstream_id":"7239","short_description":"The market for denim jeans is in equilibrium, and the price of polyester pants,…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['supply falls, increasing the price and increasing the quantity.', 'demand rises, decreasing the price and increasing the quantity.', 'demand rises, increasing the price and decreasing the quantity.', 'demand falls, decreasing the price and increasing the quantity.', 'supply rises, decreasing the price and increasing the quantity.', 'demand rises, increasing the price and increasing the quantity.', 'supply rises, increasing the price and decreasing the quantity.', 'supply falls, increasing the price and decreasing the quantity.', 'demand falls, increasing the price and decreasing the quantity.', 'supply falls, decreasing the price and decreasing the quantity.']\",\"question\":\"The market for denim jeans is in equilibrium, and the price of polyester pants, a substitute good, rises. In the jean market\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}