{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e6e79b9c-d797-59ff-87cd-a07283c05b6b","task_key":"default--test--519","task_revision_id":"1","upstream_id":"519","short_description":"A man shipped his car to Israel. In the United States his car averaged 10 miles…","config":"default","split":"test","body":"{\"category\":\"business\",\"options\":\"['7.2 kilometers per liter', '6.5 kilometers per liter', '3.7 kilometers per liter', '2.8 kilometers per liter', '3.0 kilometers per liter', '8.1 kilometers per liter', '4.16 kilometers per liter', '2.2 kilometers per liter', '9.4 kilometers per liter', '5.3 kilometers per liter']\",\"question\":\"A man shipped his car to Israel. In the United States his car averaged 10 miles to the gallon of gas. He now wishes to convert this figure in order to compare his car's efficiency to that of the Israeli cars. Convert the miles per gallon to kilometers per liter.\",\"src\":\"stemez-Business\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}