{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8b8e61fc-ab56-581c-98c7-4438f1649a10","task_key":"default--test--6565","task_revision_id":"1","upstream_id":"6565","short_description":"In which one of the following circumstances will the prevalence of a disease in…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['If recovery of the disease is faster.', 'If the incidence rate of the disease falls.', 'If the population in which the disease is measured increases.', 'If survival time with the disease increases.', 'If the disease becomes less contagious.', \\\"If the disease's incubation period decreases.\\\", 'If effective treatment for the disease becomes widely available.', 'If vaccination against the disease becomes more common.', 'If the disease mutates to a less harmful form.', \\\"If the population's overall health improves.\\\"]\",\"question\":\"In which one of the following circumstances will the prevalence of a disease in the population increase, all else being constant?\\n\",\"src\":\"ori_mmlu-nutrition\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}