{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8e63c9fc-33f3-5e3d-bcc4-b2cd5fcf0547","task_key":"default--test--2236","task_revision_id":"1","upstream_id":"2236","short_description":"Describe three processes which are considered to be instrumental in the learning…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Direct teaching, Punishment, and Extrinsic motivation', 'Cognitive dissonance, Role-playing, and Observational learning', 'Trial and error, Social comparison, and Imitation', 'Exposure, Reinforcement, and Social learning', 'Positive reinforcement, Direct teaching, and Emotional conditioning', 'Vicarious experiences, Associative learning, and Negative reinforcement', 'Habituation, Sensory adaptation, and Extrinsic motivation', 'Observational learning, Direct instruction, and Punishment', 'Shaping, Systematic desensitization, and Vicarious reinforcement', 'Imitation or modeling, classical conditioning, and operant conditioning']\",\"question\":\"Describe three processes which are considered to be instrumental in the learning of attitudes.\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}