{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"25108f06-c54b-5127-ac31-65ffecb5c961","task_key":"default--test--815","task_revision_id":"1","upstream_id":"815","short_description":"A father, policy to age 50, takes out a 5-year term insurance policyto ensure…","config":"default","split":"test","body":"{\"category\":\"business\",\"options\":\"['$100.95, $1009.50, $0, $15000', '$150, $1800, $15,000, $0', '$100.95, $1209.50, $15,000, $15,000', '$142.95, $1429.50, $0, $15,000', '$150, $1500, $0, $15,000', '$142.95, $1429.50, $15,000, $0', '$120.50, $1205.00, $0, $15,000', '$120.50, $1446.00, $15,000, $0', '$142.95, $1715.40, $15,000, $0', '$100.95, $1209.50, $15000, $0']\",\"question\":\"A father, policy to age 50, takes out a 5-year term insurance policyto ensure his son's education. (a) How much is his semiannual premium for $15,000 worth ofinsurance? (b) How much will hepayin over the five years? (c) How much will he receive at the end of the five years? (d) If he died at age 51, what would his beneficiary receive?\",\"src\":\"stemez-Business\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}