{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e5be4714-bbd6-597b-ab86-3fd7394f7ff1","task_key":"default--test--4514","task_revision_id":"1","upstream_id":"4514","short_description":"A student performs five titrations and obtains a mean result of 0.110 M, with a…","config":"default","split":"test","body":"{\"category\":\"chemistry\",\"options\":\"['Moderately accurate and precise', 'Accurate but not precise', 'Precise but not accurate', 'Both accurate and precise', 'Neither precise nor slightly accurate', 'Neither accurate nor precise', 'Highly accurate but imprecise', 'Accurate but imprecise', 'Precise and slightly accurate']\",\"question\":\"A student performs five titrations and obtains a mean result of 0.110 M, with a standard deviation of 0.001 M. If the actual concentration of the titrated solution is 0.100 M, which of the following is true about the titration results?\",\"src\":\"ori_mmlu-college_chemistry\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}