{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"00fc585e-e538-54a2-8cb1-b176da0f96e0","task_key":"default--test--4516","task_revision_id":"1","upstream_id":"4516","short_description":"A buffer is made from equal concentrations of a weak acid and its conjugate…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['It causes the pH to fluctuate unpredictably.', \\\"It causes the pH to equalize with the water's pH.\\\", 'It causes the pH to become extremely basic.', 'It neutralizes the pH completely.', 'It significantly decreases the pH.', 'It has little effect.', 'It significantly increases the pH.', 'It doubles the pH value.', 'It changes the pH asymptotically to the pKa of the acid.', 'It causes the pH to become extremely acidic.']\",\"question\":\"A buffer is made from equal concentrations of a weak acid and its conjugate base. Doubling the volume of the buffer solution by adding water has what effect on its pH?\",\"src\":\"ori_mmlu-college_chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}