{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ad7c84d9-8799-5c64-ac31-ee8f8f7af301","task_key":"default--test--11383","task_revision_id":"1","upstream_id":"11383","short_description":"(a) A 60-cycle alternator has 2 poles. What is the speedof thealternator? (b) A…","config":"default","split":"test","body":"{\"category\":\"engineering\",\"options\":\"['(a) 180 rpm (b) 6 poles', '(a) 1800 rpm (b) 30 poles', '(a) 360 rpm (b) 20 poles', '(a) 7200 rpm (b) 120 poles', '(a) 2400 rpm (b) 80 poles', '(a) 3600 rpm (b) 60 poles', '(a) 600 rpm (b) 10 poles', '(a) 900 rpm (b) 15 poles', '(a) 1200 rpm (b) 40 poles', '(a) 3000 rpm (b) 50 poles']\",\"question\":\"(a) A 60-cycle alternator has 2 poles. What is the speedof thealternator? (b) A 60-cycle alternator has a speed of 120 rpm. How many poleshas it?\",\"src\":\"stemez-ElectricalMachines\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}