{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"21c9a2f0-c2be-5740-bd5b-2707a41c6992","task_key":"default--test--9504","task_revision_id":"1","upstream_id":"9504","short_description":"A camera photographs a 120-W sodium lamp 100 m away, thefilm being exposed for…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['4.11 × 10^10 photons', '9.42 × 10^9 photons', '1.23 × 10^10 photons', '5.78 × 10^10 photons', '6.84 × 10^10 photons', '2.67 × 10^10 photons', '3.55 × 10^10 photons', '1.02 × 10^11 photons', '8.25 × 10^9 photons', '7.20 × 10^10 photons']\",\"question\":\"A camera photographs a 120-W sodium lamp 100 m away, thefilm being exposed for 1/100s. The aperture of the camerahas a diameter of 4cm, How many photons enter the cameraduring the exposure? The wavelength of sodium lightis 5893 \\\\AA, and all the energy of the lamp is assumed to beemitted as light.\",\"src\":\"stemez-Physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}