{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e9af6f87-9192-5dcd-8272-54a78aebb4d8","task_key":"default--test--9226","task_revision_id":"1","upstream_id":"9226","short_description":"On a day when the speed of sound is the fundamental frequency of a particular…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['0.65', '0.75', '0.47', '0.42', '0.55', '0.30', '0.35', '0.60', '0.40', '0.52']\",\"question\":\"On a day when the speed of sound is the fundamental frequency of a particular stopped organ pipe is 220 Hz. The second overtone of this pipe has the same wavelength as the third harmonic of an open pipe. How long is the open pipe? (Unit: m)\",\"src\":\"theoremQA-Physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}