{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"404fb237-0232-531e-8dfa-a33c1c34e9dc","task_key":"default--test--2657","task_revision_id":"1","upstream_id":"2657","short_description":"One month before finals, Conrad makes a study schedule and begins to review his…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"[\\\"Erikson's stages of psychosocial development\\\", \\\"Thorndike's law of effect\\\", \\\"Selye's general adaptation syndrome\\\", 'Yerkes-Dodson law', \\\"Bandura's social learning theory\\\", \\\"Freud's psychosexual stages of development\\\", \\\"Pavlov's classical conditioning\\\", \\\"Festinger's cognitive dissonance theory\\\", \\\"Maslow's hierarchy of needs\\\", \\\"Piaget's stages of cognitive development\\\"]\",\"question\":\"One month before finals, Conrad makes a study schedule and begins to review his notes. Two weeks before finals, Conrad is studying for hours each day to prepare. Right after finals at the start of summer vacation, Conrad comes down with the flu. Which of the following theories best explains this chain of events?\",\"src\":\"ori_mmlu-high_school_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}