{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9bb2f113-c61a-5ec2-b14e-5d205810b8f3","task_key":"default--test--2045","task_revision_id":"1","upstream_id":"2045","short_description":"What 3 factors are involved in our ability to localize sounds in space?","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['time factors, intensity factors, and psychological factors', 'Temporal factors, loudness factors, and mental factors', 'Duration factors, spatial factors, and perceptual factors', 'Echo factors, intensity factors, and sensory factors', 'Pitch factors, dynamic factors, and cognitive factors', 'Amplitude factors, directional factors, and physiological factors', 'time factors, volume factors, and emotional factors', 'Frequency factors, volume factors, and physical factors', 'time factors, intensity factors, and physical factors', 'frequency factors, intensity factors, and psychological factors']\",\"question\":\"What 3 factors are involved in our ability to localize sounds in space?\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}