{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a06c1cb9-6c19-5242-b5c6-f24a56c857bf","task_key":"default--test--12211","task_revision_id":"1","upstream_id":"12211","short_description":"In a motor the armature resistance is 0.1 ohm. When connected across 110-volt…","config":"default","split":"test","body":"{\"category\":\"engineering\",\"options\":\"['850 rpm', '1,320 rpm', '1,000 rpm', '1,061 rpm', '1,150 rpm', '1,100 rpm', '1,080 rpm', '1,250 rpm', '900 rpm', '1,200 rpm']\",\"question\":\"In a motor the armature resistance is 0.1 ohm. When connected across 110-volt mains the armature takes 20 amp, and its speed is 1,200 rpm. Determine its speed when the armature takes 50 amp from the same mains, with the field increased 10 percent.\",\"src\":\"stemez-ElectricalMachines\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}