{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"130dc169-449c-50f8-b8e9-7e0db602d50d","task_key":"default--test--6925","task_revision_id":"1","upstream_id":"6925","short_description":"Most discussions of growth focus on the supply aspect of growth. Explain how…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['Unemployment triggers rapid technological advancement.', 'Unemployment encourages more investment in the economy.', 'Unemployment can depress growth by reducing investment and capital accumulation, slowing the introduction of new technology, and leading to a lack of skilled personnel.', 'Unemployment leads to an excess of skilled personnel.']\",\"question\":\"Most discussions of growth focus on the supply aspect of growth. Explain how unemployment due to lack of demand for goods can affect growth.\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}