{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3e146e7e-d0d1-5892-bde9-e8d2787d26be","task_key":"default--test--3901","task_revision_id":"1","upstream_id":"3901","short_description":"The flux of visible photons reaching Earth from the North Star is about $4…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['7.2 $10^3$', '2.5 $10^3$', '6.1 $10^3$', '4.4 $10^3$', '5.8 $10^3$', '6.6 $10^3$', '1.8 $10^3$', '3.6 $10^3$', '3.2 $10^3$', '4.9 $10^3$']\",\"question\":\"The flux of visible photons reaching Earth from the North Star is about $4 \\\\times 10^3 \\\\mathrm{~mm}^{-2} \\\\mathrm{~s}^{-1}$. Of these photons, 30 per cent are absorbed or scattered by the atmosphere and 25 per cent of the surviving photons are scattered by the surface of the cornea of the eye. A further 9 per cent are absorbed inside the cornea. The area of the pupil at night is about $40 \\\\mathrm{~mm}^2$ and the response time of the eye is about $0.1 \\\\mathrm{~s}$. Of the photons passing through the pupil, about 43 per cent are absorbed in the ocular medium. How many photons from the North Star are focused onto the retina in $0.1 \\\\mathrm{~s}$ ?\",\"src\":\"scibench-matter\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}