{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"936b8bea-87ba-52e9-9170-beb7a13f688d","task_key":"default--test--2330","task_revision_id":"1","upstream_id":"2330","short_description":"Research on women who have had abortions indicates that","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['most women regret their decision immediately after the procedure', 'the experience frequently has serious psychological consequences, usually in the form of guilt and depression', 'women who have abortions are more likely to have difficulty conceiving in the future', 'the experience often leads to long-term physical health issues', 'the psychological consequences are no worse than they are for women who have chosen to continue their pregnancies and give birth', 'most women who have abortions think no more about the event than they would about having a wisdom tooth removed', 'subsequent problems in sexual adjustment and in relationships with men are common', 'women who have abortions are less likely to want children in the future', 'the majority of women experience severe post-traumatic stress disorder', 'most women who have abortions suffer from severe depression and anxiety for years afterward']\",\"question\":\"Research on women who have had abortions indicates that\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}