{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f081777c-bab3-599f-beae-7b8b7d9d9c17","task_key":"default--test--6878","task_revision_id":"1","upstream_id":"6878","short_description":"Consumer preferences determine what will be produced in purecapitalism. For what…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Goods and services that can be provided on an individual basis', 'Luxury goods and services', 'Goods and services which cannot be provided on an individual basis', 'Perishable goods and services with short-term utility', 'Public goods and services that are non-excludable and non-rivalrous', 'Non-essential goods and services that are overproduced', 'Goods and services that require significant government regulation', 'Goods and services with high demand', 'Basic necessities during times of crisis or shortage', 'Goods and services that have externalities affecting third parties']\",\"question\":\"Consumer preferences determine what will be produced in purecapitalism. For what types of goods might this system ofexpressing individual wants be inadequate?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}