{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"96f4a56c-9c37-5e8c-b115-7c79e5807f18","task_key":"default--test--12111","task_revision_id":"1","upstream_id":"12111","short_description":"12 gal/min of a liquid with a specific gravity of 1.3 is to be lifted with an…","config":"default","split":"test","body":"{\"category\":\"engineering\",\"options\":\"['2.0 hp', '0.8 hp', '1.8 hp', '0.5 hp', '1.2 hp', '2.5 hp', '3.0 hp', '1.5 hp', '0.6 hp', '1.0 hp']\",\"question\":\"12 gal/min of a liquid with a specific gravity of 1.3 is to be lifted with an air pump. If air can be obtained at 55 psig, and the liquid is to be raised 75 ft., determine the power needed using isentropic air compression. The pump has an efficiency of 30%.\",\"src\":\"stemez-TransportPhenomena\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}