{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f4a87cbf-2b7b-5359-9c20-448eae30155b","task_key":"default--test--4415","task_revision_id":"1","upstream_id":"4415","short_description":"If a binary component system containing initially 3g of component A and 7g of…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['m_alpha = 4.444g, m_beta = 6.556g', 'm_alpha = 5.000g, m_beta = 5.000g', 'm_alpha = 6.000g, m_beta = 4.000g', 'm_alpha = 2.500g, m_beta = 7.500g', 'm_alpha = 3.000g, m_beta = 7.000g', 'm_alpha = 1.500g, m_beta = 8.500g', 'm_alpha = 5.556g, m_beta = 4.444g', 'm_alpha = 4.444g, m_beta = 5.556g', 'm_alpha = 3.444g, m_beta = 6.556g', 'm_alpha = 2.222g, m_beta = 7.778g']\",\"question\":\"If a binary component system containing initially 3g of component A and 7g of component B is brought to a temper-ature such that two phases are present, one with a wt % of component A = 5% and the other with wt % of component A = 50%, calculate the masses of the two phases.\",\"src\":\"stemez-PhysicalChemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}