{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1c582477-1675-58a7-8b95-405e7d0e2d07","task_key":"default--test--9986","task_revision_id":"1","upstream_id":"9986","short_description":"A crude telescope is constructed of two spectacle lenses of focal lengths 100 cm…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['Angular magnification is 5, Image height is 2 inches', 'Angular magnification is 5, Image height is 1.5 inches', 'Angular magnification is 10, Image height is 1.5 inches', 'Angular magnification is 3, Image height is 1 inch', 'Angular magnification is 8, Image height is 2.5 inches', 'Angular magnification is 2, Image height is 1.5 inches', 'Angular magnification is 4, Image height is 3 inches', 'Angular magnification is 10, Image height is 2 inches', 'Angular magnification is 2, Image height is 2 inches', 'Angular magnification is 5, Image height is 0.5 inches']\",\"question\":\"A crude telescope is constructed of two spectacle lenses of focal lengths 100 cm and 20 cm respectively, a) Find its angular magnification. b) Find the height of the image formed by the objective of a building 200 ft high and one mile distant.\",\"src\":\"stemez-Optics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}