{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6e9ad7d6-57aa-58ed-8399-6840da7c9de5","task_key":"default--test--3930","task_revision_id":"1","upstream_id":"3930","short_description":"Silver bromide, AgBr, used in photography, may be prepared from AgNO_3 and NaBr.…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['50 lbs of NaBr and 85 lbs of AgNO3', '45 lbs of NaBr and 75 lbs of AgNO3', '52.7 lbs of NaBr and 84.7 lbs of AgNO3', '70 lbs of NaBr and 50 lbs of AgNO3', '40 lbs of NaBr and 90 lbs of AgNO3', '65 lbs of NaBr and 60 lbs of AgNO3', '60 lbs of NaBr and 70 lbs of AgNO3', '58 lbs of NaBr and 78 lbs of AgNO3', '48 lbs of NaBr and 88 lbs of AgNO3', '55 lbs of NaBr and 80 lbs of AgNO3']\",\"question\":\"Silver bromide, AgBr, used in photography, may be prepared from AgNO_3 and NaBr. Calculate the weight of each required for producing 93.3 lb of AgBr. (1 lb = 454 g.)\",\"src\":\"stemez-Chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}