{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7892e99e-dc7a-5c24-aa66-1bf0557597c5","task_key":"default--test--9888","task_revision_id":"1","upstream_id":"9888","short_description":"A house near a lake is kept warm by a heat engine. In winter, water from beneath…","config":"default","split":"test","body":"{\"category\":\"physics\",\"options\":\"['125 liters/s', '150 liters/s', '90 liters/s', '75 liters/s', '102.4 liters/s', '60 liters/s', '250 liters/s', '50 liters/s', '180 liters/s', '200 liters/s']\",\"question\":\"A house near a lake is kept warm by a heat engine. In winter, water from beneath the ice covering the lake is pumped through the heat engine. Heat is extracted until the water is on the point of freezing when it is ejected. The outside air is used as a sink. Assume that the air temperature is -15 °C and the temperature of the water from the lake is 2°C. Calculate the rate at which water must be pumped to the engine. The efficiency of the engine is one-fifth that of a Carnot engine and the house requires 10 kW.\",\"src\":\"stemez-Physics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}