{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"946815fc-d1a6-5f75-998a-a54ebed4a909","task_key":"default--test--12136","task_revision_id":"1","upstream_id":"12136","short_description":"The temperature inside a Carnot refrigerator is maintained at 5°C by rejecting…","config":"default","split":"test","body":"{\"category\":\"engineering\",\"options\":\"['65.7%', '110.4%', '75.3%', '78.3%', '120.8%', '89.9%', '94.5%', '102.1%', '85.2%', '99.7%']\",\"question\":\"The temperature inside a Carnot refrigerator is maintained at 5°C by rejecting heat to the surroundings at an ambient temperatureof 27°C. The inside temperature is now decreasedto-13°C, the ambient temperature remaining - constantat 27°C. Determine the percent increase in work inputfor the same quantity of heat removed.\",\"src\":\"stemez-Thermodynamics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}