{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"03a445ca-8e8f-50ff-ac8b-00cfa5e00f40","task_key":"default--test--2756","task_revision_id":"1","upstream_id":"2756","short_description":"As used in personnel selection, “adverse impact” means that","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['the total selection process results in equal results for all groups covered by law', 'a group covered by law is hired at a rate more than 80% of that of the group with the best selection', 'adverse conditions exist for selection of groups covered bylaw', 'a group not covered by law is hired at a higher rate than that of the group with the best selection', 'a group covered by law is hired at a rate less than 80% of that of the group with the best selection', 'a group not covered by law is hired at a rate less than 80% of that of the group with the best selection', 'the total selection process results in unequal results for one or more groups covered by law', 'selection interviews are “adverse” for groups covered by law and result in differential selection rates', 'adverse conditions exist for selection of groups not covered by law', 'selection interviews are “favorable” for groups covered by law and result in differential selection rates']\",\"question\":\"As used in personnel selection, “adverse impact” means that\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}