{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5c88983e-ecc1-529c-8657-966fb5dd6b63","task_key":"default--test--4269","task_revision_id":"1","upstream_id":"4269","short_description":"A manometer consists of a U-shaped tube containing a liquid. One side is…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['102$\\\\\\\\mathrm{kPa}$', '88 kPa', '130 kPa', '120 kPa', '97 $\\\\\\\\mathrm{kPa}$', '115 $\\\\\\\\mathrm{kPa}$', '85 $\\\\\\\\mathrm{kPa}$', '110 kPa', '92 kPa', '105 kPa']\",\"question\":\"A manometer consists of a U-shaped tube containing a liquid. One side is connected to the apparatus and the other is open to the atmosphere. The pressure inside the apparatus is then determined from the difference in heights of the liquid. Suppose the liquid is water, the external pressure is 770 Torr, and the open side is $10.0 \\\\mathrm{cm}$ lower than the side connected to the apparatus. What is the pressure in the apparatus? (The density of water at $25^{\\\\circ} \\\\mathrm{C}$ is $0.99707 \\\\mathrm{g} \\\\mathrm{cm}^{-3}$.)\",\"src\":\"scibench-atkins\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}