{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0fc46915-1a03-562f-9d97-7bff1dc809d3","task_key":"default--test--8783","task_revision_id":"1","upstream_id":"8783","short_description":"In a clinical trial, 30 sickle cell anemia patients are randomly assigned to two…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['a completely randomized design, with the currently marketed medicine and the experimental medicine as two treatments', 'a completely randomized design, with the currently marketed medicine and the experimental medicine as two blocks', 'a matched-pairs design, with the currently marketed medicine and the experimental medicine forming a pair', 'a double-blind design, with the currently marketed medicine and the experimental medicine as two treatments', 'a factorial design, with the currently marketed medicine and the experimental medicine as two factors', 'a randomized block design, with the currently marketed medicine and the experimental medicine as two blocks', 'a crossover design, with the currently marketed medicine and the experimental medicine as two treatments', 'a randomized block design, with the currently marketed medicine and the experimental medicine as two treatments', 'a matched-pairs design, with the currently marketed medicine and the experimental medicine as two blocks', 'a single-blind design, with the currently marketed medicine and the experimental medicine as two treatments']\",\"question\":\"In a clinical trial, 30 sickle cell anemia patients are randomly assigned to two groups. One group receives the currently marketed medicine, and the other group receives an experimental medicine. Each week, patients report to the clinic where blood tests are conducted. The lab technician is unaware of the kind of medicine the patient is taking. This design can be described as\",\"src\":\"ori_mmlu-high_school_statistics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}