{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"13a8fbeb-683e-59a5-9717-ff6006ad42fe","task_key":"default--test--3805","task_revision_id":"1","upstream_id":"3805","short_description":"In acid solution the bromate ion, BrO3- can react with other substances,…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['6 electrons on the right', '3 electrons on the right', '10 electrons on the right', '10 electrons on the left', '5 electrons on the right', '3 electrons on the left', '6 electrons on the left', '2 electrons on the left', '5 electrons on the left', '2 electrons on the right']\",\"question\":\"In acid solution the bromate ion, BrO3- can react with other substances, resulting in Br2. Balance the half-reaction for bromate ions forming bromine. The balanced half-reaction has\",\"src\":\"ori_mmlu-high_school_chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}