{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e2710a47-d5b6-5f4c-a1d5-975ee60861e4","task_key":"default--test--6566","task_revision_id":"1","upstream_id":"6566","short_description":"In which one of the following circumstances will the prevalence of a disease in…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['If the incidence rate of the disease falls.', 'If effective treatment for the disease becomes widely available.', 'If vaccination against the disease becomes more common.', 'If the disease mutates to a less harmful form.', \\\"If the population's overall health improves.\\\", 'If the disease becomes less contagious.', \\\"If the disease's incubation period decreases.\\\", 'If the population in which the disease is measured increases.', 'If recovery of the disease is faster.', 'If survival time with the disease increases.']\",\"question\":\"In which one of the following circumstances will the prevalence of a disease in the population increase, all else being constant?\\n \",\"src\":\"ori_mmlu-nutrition\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}