{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"653ded84-9875-50e7-ba3a-bc6b82e8e359","task_key":"default--test--2311","task_revision_id":"1","upstream_id":"2311","short_description":"What are three factors that influence our olfactory experience?","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['Temperature, humidity, and light intensity', 'Sound intensity, pitch, and duration', 'Color, texture, and size of the substance', 'Volume, density, and mass of the substance', 'Altitude, air pressure, and geographic location', 'Quality, concentration, and rate of flow of the stimulus', 'Dietary habits, exercise frequency, and sleep patterns']\",\"question\":\"What are three factors that influence our olfactory experience?\",\"src\":\"stemez-Psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}