{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a671908a-a3ec-5022-a3f2-fbaac11841bb","task_key":"default--test--2115","task_revision_id":"1","upstream_id":"2115","short_description":"If, after a conditioned response has been acquired, the procedureis changed so…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Intermittent reinforcement of the conditioned response', 'Habituation to the conditioned stimulus', 'Transformation of conditioned respondent behavior', 'Emergence of a new unconditioned response', 'No change in conditioned respondent behavior', 'Increased resistance to extinction', 'Strengthening of conditioned respondent behavior', 'Spontaneous recovery of the conditioned response', 'Extinction of conditioned respondent behavior', 'Generalization of the conditioned response to more stimuli']\",\"question\":\"If, after a conditioned response has been acquired, the procedureis changed so that the CS is repeatedly presented withoutbeing followed by the UCS, what is the result ?\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}