{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"06245f0e-ddd4-53dc-8ead-c25e267cb332","task_key":"default--test--9649","task_revision_id":"1","upstream_id":"9649","short_description":"A student drops a water-filled balloon from the roof of the tallest building in…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['71$\\\\\\\\mathrm{~m}$ ', '65 $\\\\\\\\mathrm{~m}$', '55 m', '80 $\\\\\\\\mathrm{~m}$', '78 m', '85 m', '75 m', '90 m', '50 m', '60 $\\\\\\\\mathrm{~m}$']\",\"question\":\"A student drops a water-filled balloon from the roof of the tallest building in town trying to hit her roommate on the ground (who is too quick). The first student ducks back but hears the water splash $4.021 \\\\mathrm{~s}$ after dropping the balloon. If the speed of sound is $331 \\\\mathrm{~m} / \\\\mathrm{s}$, find the height of the building, neglecting air resistance.\",\"src\":\"scibench-class\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}