{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2037c634-fe3d-5e80-b0aa-b5d386e87d58","task_key":"default--test--734","task_revision_id":"1","upstream_id":"734","short_description":"(a) 10 annual mortgage payments of $1,000, (b) 12 monthly payments of $100 on…","config":"default","split":"test","body":"{\"category\":\"business\",\"options\":\"['$1,225.00', '$1,500.00', '$1,580.42', '$1,250.00', '$1,450.00', '$1,375.69', '$1,600.00', '$1,700.00', '$1,305.75', '$1,520.34']\",\"question\":\"(a) 10 annual mortgage payments of $1,000, (b) 12 monthly payments of $100 on his automobile, (c) a bill for $2,000 due in two years, (d) a bill for $1,000 due today. Using an annual interest rate of 12 percent (nominal rate on the automobile loan and effective rate on all other debts), determine the annual amount necessary to retire the entire debt in 15 years.\",\"src\":\"stemez-Business\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}