{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c1d9eb36-b4ff-59fa-8df8-86965e7121b9","task_key":"default--test--4634","task_revision_id":"1","upstream_id":"4634","short_description":"In over 90% of cases, the concentration of ethanol (C_2H_5OH, density = 0.80…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['30 ml of 80 proof Scotch whiskey', '100 ml of 80 proof Scotch whiskey', '110 ml of 80 proof Scotch whiskey', '120 ml of 80 proof Scotch whiskey', '150 ml of 80 proof Scotch whiskey', '65 ml of 80 proof Scotch whiskey', '50 ml of 80 proof Scotch whiskey', '75 ml of 80 proof Scotch whiskey', '45 ml of 80 proof Scotch whiskey', '88 ml of 80 proof Scotch whiskey']\",\"question\":\"In over 90% of cases, the concentration of ethanol (C_2H_5OH, density = 0.80 g/ml) in blood necessary to produce intoxication is 0.0030 g/ml. A concentration of 0.0070 g/ml is fatal. What volume of 80 proof (40% ethanol by volume) Scotch whiskey must an intoxicated person consume before the concentration of ethanol in his blood reaches a fatal level? Assume that all the alcohol goes directly to the blood and that the blood volume of a person is 7.0 liters.\",\"src\":\"stemez-Chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}