{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"df07ca91-e811-5be4-a1f0-074267bef2ef","task_key":"default--test--2052","task_revision_id":"1","upstream_id":"2052","short_description":"How is incoming sensory verbal information abstracted in terms of…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['The nervous system classifies information based on emotional response', 'The nervous system classifies information over a period of time', 'The nervous system classifies information randomly', 'The nervous system classifies information based on the type of senses', 'The nervous system classifies information based on previous experiences only', 'The nervous system classifies information instantly', 'The nervous system classifies information using a single, universal category', 'The nervous system classifies information according to the body part receiving the stimulus', 'The nervous system does not classify information', 'The nervous system classifies information based on the intensity of the stimulus']\",\"question\":\"How is incoming sensory verbal information abstracted in terms of classification?\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}