{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c6dc8c0f-4377-5902-a9a3-4c02ff6ce367","task_key":"default--test--9394","task_revision_id":"1","upstream_id":"9394","short_description":"Delivery trucks which operate by making use of the energy stored in a rotating…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['80 min', '105 min', '75 min', '120 min', '70 min', '110 min', '85 min', '94.2 min', '100 min', '130 min']\",\"question\":\"Delivery trucks which operate by making use of the energy stored in a rotating flywheel have been in use for some time in Germany. The trucks are \\\"charged up\\\" before leaving by using an electric motor to get the flywheel up to its top speed of 6000 rev\\\\bulletmin^-1. If one such flywheel is a solid homogeneous cylinder of weight 1120 lb and diameter 6 ft, how long can the truck operate before returning to its base for \\\"recharging\\\\textquotedblright, if its average power requirement is 10 hp?\",\"src\":\"stemez-Physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}