{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0c5a137d-730a-5882-9bf4-096cdc1ed760","task_key":"default--test--9624","task_revision_id":"1","upstream_id":"9624","short_description":"A copper kettle, the circular bottom of which is 6.0 in. in diameter and 0.062…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['15 Btu', '25 Btu', '28 Btu', '20 Btu', '30 Btu', '18 Btu', '24 Btu', '32 Btu', '22 Btu', '26 Btu']\",\"question\":\"A copper kettle, the circular bottom of which is 6.0 in. in diameter and 0.062 in. thick, is placed over a gas flame. On assuming that the average temperature of the outer surface of the copper is 214°F and that the water in the kettle is at its normal boiling point, how much heat is conducted through the bottom in 5.0 sec? The thermal conductivity may be taken as 2480 Btu/(ft^2 hr F°/in).\",\"src\":\"stemez-Physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}