{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0f484e0d-eac7-50e7-8e29-67b527f8438a","task_key":"default--test--2065","task_revision_id":"1","upstream_id":"2065","short_description":"Aronson and Mills examined the effects severity of initiation into a group on…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['were more likely to drop out of the group.', 'rated the group as less interesting and enjoyable.', 'felt more alienated within the group.', 'felt more indifferent towards the group.', 'were less likely to contribute to group activities.', 'were more likely to assume leadership roles within the group.', 'were more likely to dislike fellow group members.', 'rated the group as more interesting and enjoyable.', 'were less likely to recommend the group to others.', 'experienced less satisfaction from group activities.']\",\"question\":\"Aronson and Mills examined the effects severity of initiation into a group on subsequent attitudes toward the group. Results of their study indicated that, compared to women who underwent a mild initiation, women who underwent a severe initiation as a precondition of group membership:\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}