{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9cceec59-02ef-5879-a39b-61c42681e0bd","task_key":"default--test--8959","task_revision_id":"1","upstream_id":"8959","short_description":"A home buyer can afford to spend no more than $\\$ 800 /$ month on mortgage…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['102,965.21 $', '$125,789.99', '$98,765.43', '$75,000.00', '$150,000.00', '$68,500.45', '$110,200.35', '$93,450.89', '$85,250.65', '$120,345.78']\",\"question\":\"A home buyer can afford to spend no more than $\\\\$ 800 /$ month on mortgage payments. Suppose that the interest rate is $9 \\\\%$ and that the term of the mortgage is 20 years. Assume that interest is compounded continuously and that payments are also made continuously.\\nDetermine the total interest paid during the term of the mortgage.\",\"src\":\"scibench-diff\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}