{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f5c6cb52-cfd7-53f1-a1da-a5de2f9b03f7","task_key":"default--test--1483","task_revision_id":"1","upstream_id":"1483","short_description":"Catharine MacKinnon argues that since men dominate women, the question is…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The power dynamic is determined by individual strength and not gender.', \\\"The idea of 'woman' needs to be redefined.\\\", 'Women inherently possess more power than men.', 'Gender-based inequality is a myth.', 'Men and women share equal power in society.', 'Equality is impossible because men define women as different.', 'Women can achieve equality through political pressure.', 'Men and women are inherently equal in all aspects.', 'The concept of power is subjective and varies from person to person.', 'The world is phallocentric and oppressive.']\",\"question\":\"Catharine MacKinnon argues that since men dominate women, the question is ultimately one of power. Which proposition below is the most inconsistent with this argument?\",\"src\":\"ori_mmlu-jurisprudence\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}