{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"14e5cf3c-50ff-5671-89d4-829ff74c9f83","task_key":"default--test--10007","task_revision_id":"1","upstream_id":"10007","short_description":"Count Rumford observed that using cannon boring machinery a single horse could…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['30$\\\\\\\\mathrm{~m}$', '20 $\\\\\\\\mathrm{~m}$', '45 $\\\\\\\\mathrm{~m}$', '25 m', '40 m', '50 m', '35 $\\\\\\\\mathrm{~m}$', '10 m', '60 m', '15 m']\",\"question\":\"Count Rumford observed that using cannon boring machinery a single horse could heat $11.6 \\\\mathrm{~kg}$ of ice water $(T=273 \\\\mathrm{~K})$ to $T=355 \\\\mathrm{~K}$ in 2.5 hours. Assuming the same rate of work, how high could a horse raise a $225 \\\\mathrm{~kg}$ weight in 2.5 minutes? Assume the heat capacity of water is $4.18 \\\\mathrm{~J} \\\\mathrm{~K}^{-1} \\\\mathrm{~g}^{-1}$.\",\"src\":\"scibench-thermo\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}