{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ac7ed8b4-c8b4-582d-94ea-92d82c06d1e6","task_key":"default--test--7615","task_revision_id":"1","upstream_id":"7615","short_description":"Define `marginalism' (as used in economics).","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Marginalism is the study of economic trends.', 'Marginalism is the method of analysis that assesses the impact of changes in production volume on total satisfaction.', 'Marginalism is the method of analysis focusing on the first unit in production.', 'Marginalism is the study of how market prices are established by the intersection of supply and demand curves.', 'Marginalism is the theory that all units in production have equal value.', 'Marginalism is the practice of increasing production by one unit to assess cost changes.', 'Marginalism is the concept that the value of a good is determined by its historical cost.', 'Marginalism is the method of analysis that concentrates attention on the last unit in production or consumption.', 'Marginalism is the method of analysis that ignores the last unit in production or consumption.', 'Marginalism is the economic principle that prioritizes the most abundant resources.']\",\"question\":\"Define `marginalism' (as used in economics).\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}