{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"abfd8701-caf6-5f83-9286-2eb80c0c316a","task_key":"default--test--6954","task_revision_id":"1","upstream_id":"6954","short_description":"What is meant by \"economic resources\" or \"factorsof production\"? Name three…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Capital, commodities, and currency', 'raw materials, human resources and technology', 'natural resources, artificial resources and human resources', 'Water, oil, and minerals', 'Intellectual property, workforce, and real estate', 'money, machines and management', 'Skills, stocks, and bonds', 'Infrastructure, entrepreneurship, and knowledge', 'land, labor and capital', 'Technology, patents, and brands']\",\"question\":\"What is meant by \\\"economic resources\\\" or \\\"factorsof production\\\"? Name three categories of resources.\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}