{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b5ecdc7f-3164-5866-86c1-f9a4dea32231","task_key":"default--test--10197","task_revision_id":"1","upstream_id":"10197","short_description":"If the sensitive plate of a pinhole camera is 20 cm from the pinhole, what…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['0.45 mm', '0.65 mm', '0.57 mm', '0.60 mm', '0.50 mm', '0.72 mm', '0.30 mm', '0.75 mm', '0.40 mm', '0.55 mm']\",\"question\":\"If the sensitive plate of a pinhole camera is 20 cm from the pinhole, what should be the diameter of the pinhole, according to Abney's formula? Assume that the photographic film used has its highest sensitivity at a wavelength of 450 nm.\",\"src\":\"stemez-Optics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}