{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b6f94eb4-da0b-5be9-9ede-6c562d789b76","task_key":"default--test--6184","task_revision_id":"1","upstream_id":"6184","short_description":"A placebo-controlled clinical trial is conducted to assess whether a new…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['The study will have less power', 'The study will require fewer participants', 'There is an increased likelihood of statistically significant findings', 'The study will be less reliable', 'Significant findings can be reported with greater confidence', 'The placebo effect will be more pronounced', 'The study will have more power', 'There is a decreased likelihood of statistically significant findings']\",\"question\":\"A placebo-controlled clinical trial is conducted to assess whether a new antihypertensive drug is more effective than standard therapy. A total of 5000 patients with essential hypertension are enrolled and randomly assigned to one of two groups: 2500 patients receive the new drug and 2500 patients receive placebo. If the alpha is set at 0.01 instead of 0.05, which of the following is the most likely result?\",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}