{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a5c5b5cf-a55a-5b56-b5f7-e92c0f9db0cc","task_key":"default--test--8382","task_revision_id":"1","upstream_id":"8382","short_description":"A virus is spreading throughout the population of a town, and the number of…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['February 10th', 'January 28th', 'January 10th', 'March 1st', 'January 15th', 'January 31st', 'March 10th', 'January 21st', 'February 21st', 'February 1st']\",\"question\":\"A virus is spreading throughout the population of a town, and the number of people who have the virus doubles every 3 days. If there are 1,000 people in the town, and 10 people have the virus on January 1st, what is the earliest date at which the entire town would be infected with the virus, given that there are 365 days in a year, and 31 days in the month of January?\",\"src\":\"ori_mmlu-high_school_mathematics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}