{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"67c9e898-8560-5b52-9e14-ab92f2281531","task_key":"default--test--9866","task_revision_id":"1","upstream_id":"9866","short_description":"If a normal eye cannot separate two objects (or images) subtending an angle of…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['660.6', '750', '500', '550', '460', '800', '1000', '720.4', '850', '900']\",\"question\":\"If a normal eye cannot separate two objects (or images) subtending an angle of less than 1.5 minutes, approximate the magnifying power that is necessary to make use (visually) of the full resolving power of a telescope whose objective has a diameter of 40 inches. (Assume a mean wavelength of 5.5 × 10^-5 cm.)\",\"src\":\"stemez-Optics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}