{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"06c5e33c-2225-589a-bb39-2bf5eb17f6a4","task_key":"default--test--535","task_revision_id":"1","upstream_id":"535","short_description":"What is the annual cost of a 20-payment life policy with a face of $4,000 for a…","config":"default","split":"test","body":"{\"category\":\"business\",\"options\":\"['Annual: $91.80, Semiannual: $88.28, Quarterly: $90.04', 'Annual: $82.50, Semiannual: $41.25, Quarterly: $20.63', 'Annual: $45.02, Semiannual: $22.95, Quarterly: $88.28', 'Annual: $75.50, Semiannual: $37.75, Quarterly: $19.00', 'Annual: $100.00, Semiannual: $51.00, Quarterly: $26.00', 'Annual: $95.76, Semiannual: $47.88, Quarterly: $23.94', 'Annual: $60.24, Semiannual: $30.12, Quarterly: $15.06', 'Annual: $88.28, Semiannual: $90.04, Quarterly: $91.80', 'Annual: $90.04, Semiannual: $91.80, Quarterly: $88.28', 'Annual: $110.32, Semiannual: $55.16, Quarterly: $27.58']\",\"question\":\"What is the annual cost of a 20-payment life policy with a face of $4,000 for a 25-year old, if the payments are made (a) annually (b) semiannually (c) quarterly?\",\"src\":\"stemez-Business\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}