{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"210cafc7-d49d-5958-99ec-10926ef43c23","task_key":"default--test--1988","task_revision_id":"1","upstream_id":"1988","short_description":"During the second stage of Kohlberg’s preconventional level of moral…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['they are taught that rules are unbreakable.', 'they enjoy following the rules.', 'they are rewarded for following rules.', 'they feel they have a personal duty to uphold rules and laws.', 'doing so helps them satisfy their personal needs.', 'they understand the concept of fairness and justice.', 'doing so helps them avoid punishment.', '\\\"\\\"\\\"everyone else is doing it.\\\"\\\"\\\"', 'they believe in the inherent goodness of rules.', 'they fear societal disapproval.']\",\"question\":\"During the second stage of Kohlberg’s preconventional level of moral development, children obey rules because:\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}