{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"87635cf0-81da-5fa2-a077-12da192350d9","task_key":"default--test--10282","task_revision_id":"1","upstream_id":"10282","short_description":"A lens found in 35 mm cameras has a focal length of 50 mm and a diameter of 25…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['200 lines per mm', '500 lines per mm', '1200 lines per mm', '1000 lines per mm', '550 lines per mm', '350 lines per mm', '800 lines per mm', '741 lines per mm', '650 lines per mm', '900 lines per mm']\",\"question\":\"A lens found in 35 mm cameras has a focal length of 50 mm and a diameter of 25 mm, corresponding to a maximum aperture of (f / 2). The resolution of a lens is commonly expressed as the maximum number of lines per millimeter in the image that can be resolved. If the resolution of the lens is diffraction-limited, approximately what is the resolution in lines per mm? (Good quality 35 mm camera lenses typically have a resolution of the order of 100 [(lines) / mm], determined by surface precision rather than diffraction.)\",\"src\":\"stemez-Optics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}