{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"abfb0581-a95c-5ac5-942e-a5f73160a6b5","task_key":"default--test--3734","task_revision_id":"1","upstream_id":"3734","short_description":"Assume all gases are perfect unless stated otherwise. Note that 1 atm = 1.013 25…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['8.5 kg', '9.3 kg', '3.7$\\\\\\\\text{kg}$', '5.6$\\\\\\\\text{kg}$', '2.5$\\\\\\\\text{kg}$', '7.85 kg', '3.0 kg', '6.2 kg', '4.09$\\\\\\\\text{kg}$ ', '1.75 kg']\",\"question\":\"Assume all gases are perfect unless stated otherwise. Note that 1 atm = 1.013 25 bar. Unless otherwise stated, thermochemical data are for 298.15 K. An average human produces about $10 \\\\mathrm{MJ}$ of heat each day through metabolic activity. Human bodies are actually open systems, and the main mechanism of heat loss is through the evaporation of water. What mass of water should be evaporated each day to maintain constant temperature?\",\"src\":\"scibench-atkins\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}