{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"791d9e2d-f7aa-5f3a-8f4d-99eb6d963ff2","task_key":"default--test--2090","task_revision_id":"1","upstream_id":"2090","short_description":"Research on explicit (intentional) and implicit (automatic) memory indicates…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['both forms of memory fluctuate in capacity throughout the adult life span', 'neither forms of memory are affected by age, only by a disease process like Alzheimer’s', 'both forms of memory show a fairly consistent, gradual decline over the adult life span', 'both forms of memory remain stable, except when affected by a disease process like Alzheimer’s', 'explicit memory capacity remains stable, while implicit memory capacity declines with age', 'implicit memory capacity declines with age, but explicit memory capacity remains stable', 'both forms of memory increase their capacity with age', 'implicit memory capacity increases with age, but explicit memory capacity remains stable', 'explicit memory capacity increases with age, while implicit memory capacity declines', 'explicit memory capacity declines with age, but implicit memory capacity remains stable']\",\"question\":\"Research on explicit (intentional) and implicit (automatic) memory indicates that\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}