{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bccc9b77-ae26-593e-9d38-8d6aff11d807","task_key":"default--test--4293","task_revision_id":"1","upstream_id":"4293","short_description":"The average concentration of particulate lead in a city with highly polluted air…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['6.4 μg', '25.0 μg', '30.0 μg', '8.5\\\\\\\\mug', '10.6 μg', '21.3 μg', '42.5\\\\\\\\mug', '15.9\\\\\\\\mug', '37.5\\\\\\\\mug', '12.7 μg']\",\"question\":\"The average concentration of particulate lead in a city with highly polluted air is 5\\\\mug/m^3. If an adult respires 8500 liters of air daily and if 50 per cent of the particles below 1\\\\mumin size are retained in the lungs, calculate how much lead is absorbed in the lungs each day. Of the particu-late lead in urban air, 75 per cent is less than 1\\\\mumin size.\",\"src\":\"stemez-Chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}