{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"123db55a-8e5e-5160-9ae1-c8f9e32cdd1e","task_key":"default--test--9664","task_revision_id":"1","upstream_id":"9664","short_description":"A laser beam (694nm wavelength) is used in measuring variations in the size of…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['1.5km', '0.5km', '0.9km', '3.0km', '2.5km', '1.22km', '0.8km', '2.0km', '0.643km', '0.3km']\",\"question\":\"A laser beam (694nm wavelength) is used in measuring variations in the size of the moon by timing its return from mirror systems on the moon. If the beam is expanded to 1m diameter and collimated, estimate its size at the moon. (Moon's distance \\\\sim 3.8 × 10^5 km.)\",\"src\":\"stemez-Optics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}