{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8096dcf4-3380-5e39-870b-b4957babd79c","task_key":"default--test--2638","task_revision_id":"1","upstream_id":"2638","short_description":"In Bandura's social learning theory, the concept of seif-reinforcement is…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['peer pressure influences decision making', 'children can overcome the negative effects of poor models', 'punishment can be avoided', 'the impact of positive reinforcement is amplified', 'learned behavior is maintained over time', 'rival responses are acquired', 'children can learn to mimic adult behavior', 'children learn to differentiate between right and wrong', 'children learn to imitate behavior without understanding its implications', 'socially appropriate behavior can be learned in the absence of models']\",\"question\":\"In Bandura's social learning theory, the concept of seif-reinforcement is critical for explaining how in the absence of external reward\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}