{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f5fd7452-907e-5849-807f-521930e690d3","task_key":"default--test--2141","task_revision_id":"1","upstream_id":"2141","short_description":"Milgram (1965) found that participants in his study were more willing to…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['bystander effect.', 'conformity.', 'psychological reactance.', 'self-perception.', 'self-verification.', 'groupthink.', 'deindividuation.', 'social facilitation.', 'cognitive dissonance.', 'pluralistic ignorance.']\",\"question\":\"Milgram (1965) found that participants in his study were more willing to administer electric shocks to another person when they couldn't see the victim and the victim couldn't see them. This finding is predicted by Zimbardo’s notion of:\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}