{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4cd072d2-6031-5e23-bc76-7b9c8fe79e21","task_key":"default--test--5115","task_revision_id":"1","upstream_id":"5115","short_description":"The City of Windemere decided to construct several large windmills to generate…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Permanent.', 'Internal Service.', 'Trust and Agency.', 'Capital projects.', 'Enterprise.', 'Debt Service.', 'General Fund.', 'Pension (and other employee benefit) Trust.', 'Investment Trust.', 'Special revenue.']\",\"question\":\"The City of Windemere decided to construct several large windmills to generate electrical power. The construction was financed through a general residential property tax levy for the next ten years. Utility revenues are intended to offset all expenses associated with the windmills. The land for the windmills was donated to the city by a local farmer. The land from the farmer should be reported in which fund type?\",\"src\":\"ori_mmlu-professional_accounting\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}