{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"cb0febd2-c881-5559-a219-b11e152ab544","task_key":"default--test--6976","task_revision_id":"1","upstream_id":"6976","short_description":"If the demand for grapes increases simultaneously with an increase in the supply…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['equilibrium quantity rises, and the price rises.', 'equilibrium quantity rises, but the price change is ambiguous.', 'equilibrium quantity stays the same, but the price rises.', 'equilibrium quantity falls, but the price rises.', 'equilibrium quantity is ambiguous, but the price rises.', 'both equilibrium quantity and price stay the same.', 'equilibrium quantity falls, and the price falls.', 'equilibrium quantity rises, but the price falls.', 'equilibrium quantity stays the same, and the price falls.', 'equilibrium quantity falls, but the price change is ambiguous.']\",\"question\":\"If the demand for grapes increases simultaneously with an increase in the supply of grapes, we can say that\",\"src\":\"ori_mmlu-high_school_microeconomics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}