{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d8257d92-b572-5f59-b4d8-7b385262dfc4","task_key":"default--test--11832","task_revision_id":"1","upstream_id":"11832","short_description":"(a )A transmission line consists of two No. 0 copper wires and is 5 miles long.…","config":"default","split":"test","body":"{\"category\":\"engineering\",\"options\":\"['29.7 kilowatts; 712,800 calories', '26 kilowatts; 695,000 calories', '31 kilowatts; 720,000 calories', '28.5 kilowatts; 715,500 calories', '30.5 kilowatts; 725,000 calories', '32 kilowatts; 730,000 calories', '30 kilowatts; 715,000 calories', '27.5 kilowatts; 705,000 calories', '29 kilowatts; 700,000 calories', '28 kilowatts; 710,000 calories']\",\"question\":\"(a )A transmission line consists of two No. 0 copper wires and is 5 miles long. How much power is wasted in the line when 75 amperes are flowing? (b) How many calories of heat per hour are developed in an electric heater which takes 7.5 amperes at 110 volts pressure?\",\"src\":\"stemez-ElectricalMachines\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}