{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4f697262-0c49-57d8-9ba8-67d259780952","task_key":"default--test--7586","task_revision_id":"1","upstream_id":"7586","short_description":"How did the government stand behind the banks through the formation of the…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['By increasing taxes on banks', 'By guaranteeing loans made by banks to businesses and individuals', 'By capping the interest rates banks can charge on loans', 'By providing financial grants to banks', 'By taking over all private banks', 'By insuring only the deposits of commercial accounts', 'By protecting all bank deposits with government insurance', 'By allowing banks to borrow unlimited funds from the government at zero interest', 'By offering a one-time bailout fund for struggling banks', 'By investing government funds directly into bank stocks']\",\"question\":\"How did the government stand behind the banks through the formation of the Federal Deposit Insurance Corporation?\",\"src\":\"stemez-Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}