{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e896e0ee-872c-5b2a-8179-3c2f737a79a0","task_key":"default--test--3849","task_revision_id":"1","upstream_id":"3849","short_description":"The pressure in an automobile tire is measured by a tire gauge in winter at 0°C…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['34 psi', '32 psi', '40 psi', '35 psi', '44 psi', '36 psi', '45 psi', '38 psi', '48 psi', '42 psi']\",\"question\":\"The pressure in an automobile tire is measured by a tire gauge in winter at 0°C as 30 pounds per square inch (psi). The gauge measures the difference between the actual tire pressure and atmospheric pressure (15 psi). Assume that the tire does not leak air and does not change its volume. What will be the new gauge pressure if the same tire is measured during the summer at 50°C?\",\"src\":\"stemez-Chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}