{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"acacb722-1e13-5748-be7a-e483b6e55dfd","task_key":"default--test--3745","task_revision_id":"1","upstream_id":"3745","short_description":"When air is let out of an automobile tire, the air feels cold to the touch.…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['The air feels cold because it mixes with colder atmospheric air', 'The air feels cold due to a chemical reaction within the tire', 'The air temperature falls because it is doing work and expending energy', \\\"The cooling sensation is a result of the air's rapid expansion\\\", \\\"The cooling sensation is due to the air's humidity\\\", 'The cooling is caused by the evaporation of water on the skin', \\\"The cooling sensation is due to the air's decreased velocity upon exiting the tire\\\", 'The air feels cold because it is under high pressure', \\\"The air feels cold because of the tire's insulation properties\\\", \\\"The air feels cold due to the tire's heat absorption\\\"]\",\"question\":\"When air is let out of an automobile tire, the air feels cold to the touch. Explain why you feel this cooling sensation.\",\"src\":\"stemez-Chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}