{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"370b0dc3-e12f-5cb7-a243-ea66f693dc1b","task_key":"default--test--6555","task_revision_id":"1","upstream_id":"6555","short_description":"A physician is conducting a retrospective review of a trial involving the use of…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Individuals who take Drug X have two times the risk of dying from this type of cancer', 'Individuals who take Drug X have four times the risk of dying from this type of cancer', 'Individuals who take Drug X have three times the risk of dying from this type of cancer', 'Individuals who take Drug X have six times the risk of dying from this type of cancer', 'Individuals who do not take Drug X have three times the risk of dying from this type of cancer', 'Individuals who take Drug X have half the risk of dying from this type of cancer', 'Individuals who take Drug X have an equal risk of dying from this type of cancer', 'Individuals who do not take Drug X have two times the risk of dying from this type of cancer', 'Individuals who take Drug X have five times the risk of dying from this type of cancer', 'Individuals who take Drug X have zero risk of dying from this type of cancer']\",\"question\":\"A physician is conducting a retrospective review of a trial involving the use of Drug X in patients with a specific disease. It is known that Drug X is associated with an increased probability of cancer in patients who use the drug. A total of 600 individuals with a specific disease were included in the trial. Of the participants, 200 individuals received Drug X and 400 individuals did not receive it. One hundred individuals who received Drug X died of a particular type of cancer and 100 individuals who did not receive the drug died of the same type of cancer. Based on these data, which of the following is the relative risk of death from this type of cancer in individuals who take Drug X as compared with individuals who do not take Drug X?\",\"src\":\"ori_mmlu-professional_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}