{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e89f4c99-218b-5d95-9730-7c6acffe17a5","task_key":"default--test--10400","task_revision_id":"1","upstream_id":"10400","short_description":"Given 2 colors whose HSI representations are given as follows: (a) $(pi,…","config":"default","split":"test","body":"{\"category\":\"computer science\",\"options\":\"['Brightness cannot be compared without additional information about the viewing conditions', '(b) is brighter due to higher hue value', 'Cannot be determined from the given data', 'Brightness is subjective and cannot be determined from HSI values alone', '(b) is brighter because it has a higher saturation and lower hue value', '(a)', 'Both are equally bright', '(b)']\",\"question\":\"Given 2 colors whose HSI representations are given as follows: (a) $(pi, 0.3,0.5)$, (b) $(0.5 pi, 0.8,0.3)$, which color is brighter?\",\"src\":\"theoremQA-EECS\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}