{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0eff6e0b-0658-5a69-812f-5f037abd84cd","task_key":"default--test--2197","task_revision_id":"1","upstream_id":"2197","short_description":"Behaviorist views have been criticized by other approaches for a number of…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['the focus on negative reinforcement', 'the emphasis on observation over experimentation', 'the use of the trial-and-error system', 'the application of cognitive theory', 'the disregard for individual differences in behavior', 'the use of the operant conditioning principle', 'the use of the animals in laboratory experiments', 'the use of the classical conditioning principle', 'the use of the humans in laboratory experiments']\",\"question\":\"Behaviorist views have been criticized by other approaches for a number of reasons. Ethologists argued that the behaviorists ignored innate, built-in biases in learning due to evolution, but also disagreed with:\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}