{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"eb952f39-5e44-5924-911d-f5969ceaf555","task_key":"default--test--2072","task_revision_id":"1","upstream_id":"2072","short_description":"When using the Premack Principle to modify a behavior, the reinforcer is:","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['a verbal praise or encouragement.', 'a behavior that occurs frequently.', 'a specific reward given after the desired behavior.', 'a stimulus that is artificially created.', 'a generalized conditioned reinforcer.', 'a stimulus that naturally elicits the desired behavior.', 'a behavior that occurs rarely.', 'a negative reinforcement.', 'a punishment that follows the undesired behavior.', 'applied intermittently.']\",\"question\":\"When using the Premack Principle to modify a behavior, the reinforcer is:\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}