{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"69d248e1-44b3-5012-ae29-b546d9d91a39","task_key":"default--test--2456","task_revision_id":"1","upstream_id":"2456","short_description":"Attempting to teach a new behavior through the use of aversive techniques is…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['The behavior becomes more frequent and intense', 'Emotional reactivity is often increased', 'The frequency of behavior is often low', \\\"It increases the individual's motivation to learn new behaviors\\\", 'The individual develops a positive association with the aversive technique', 'Inhibition co conditioning is encountered', 'It leads to an immediate and permanent change in behavior']\",\"question\":\"Attempting to teach a new behavior through the use of aversive techniques is difficult because of which of the following\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}