{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9577aa74-2c91-5690-8b74-a1c91ba4fa0e","task_key":"default--test--7587","task_revision_id":"1","upstream_id":"7587","short_description":"Is pure capitalism adaptable to changes in consumer tastes?","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Pure capitalism adapts to changes in consumer tastes by adjusting production of consumer goods and resource markets.', 'Pure capitalism disregards changes in resource markets.', 'Pure capitalism does not adjust production based on consumer tastes.', 'Pure capitalism is immune to changes in consumer tastes.']\",\"question\":\"Is pure capitalism adaptable to changes in consumer tastes?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}