{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"de38400b-5a85-50b1-9a08-f1f4bd9e967f","task_key":"default--test--9601","task_revision_id":"1","upstream_id":"9601","short_description":"An electric kettle contains 2 liters of water which it heats from 20°C to…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['(a) 3 kW, (b) 3 cents, (c) 20 Ohms, (d) 12 A', '(a) 2.5 kW, (b) 2.5 cents, (c) 18.5 Ohms, (d) 11.5 A', '(a) 1.8 kW, (b) 2.16 cents, (c) 22 Ohms, (d) 9.1 A', '(a) 1.5 kW, (b) 1.5 cents, (c) 15 Ohms, (d) 10 A', '(a) 2 kW, (b) 2.4 cents, (c) 16 Ohms, (d) 10.5 A', '(a) 2.23 kW, (b) 2.23 cents, (c) 17.9 Ohms, (d) 11.2 A', '(a) 1.75 kW, (b) 2.1 cents, (c) 19 Ohms, (d) 11.8 A', '(a) 2.1 kW, (b) 2.52 cents, (c) 18 Ohms, (d) 10.8 A', '(a) 2.8 kW, (b) 3.36 cents, (c) 14 Ohms, (d) 12.5 A', '(a) 2.45 kW, (b) 2.94 cents, (c) 15.5 Ohms, (d) 11 A']\",\"question\":\"An electric kettle contains 2 liters of water which it heats from 20°C to boiling point in 5 min. The supply voltage is 200 V and a kWh (kilowatt-hour) unit costs 2 cents. Calculate (a) the power consumed (assume that heat losses are negligible),(b) the cost of using the kettle under these conditions six times,(c) the re-sistance of the heating element, and (d) the current in the element.\",\"src\":\"stemez-Physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}