{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"04d7589b-4921-5753-af80-d4919c880aa2","task_key":"default--test--7801","task_revision_id":"1","upstream_id":"7801","short_description":"A dentist has noticed that about two children in every seven whom he sees…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['P(X = 3 out of 5)', 'P(X = 0)', 'P(X = 7 out of 10)', 'P(X = 1)', '1 – P(X = 1)', 'P(X = 1, 2, 3, 4, 5)', 'P(X = 2 out of 7)', 'P(X = 2, 3, 4, 5, 6, 7)', '1 – P(X = 0)', 'P(X = 2 out of 5)']\",\"question\":\"A dentist has noticed that about two children in every seven whom he sees professionally develop cavities before they turn 10 years old. Last week he examined the teeth of five unrelated children younger than 10. Let X be the number of children who develop cavities before turning 10. Which of the following gives the probability that at least one will develop a cavity before turning 10?\",\"src\":\"ori_mmlu-high_school_statistics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}