{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7587c5f1-70c8-57db-a791-3a9c6f65bb2a","task_key":"default--test--8906","task_revision_id":"1","upstream_id":"8906","short_description":"Which pair of expressions is equivalent for any variable value greater than…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['4n + 2o and 8no', '2j + 2j + k and 4jk', '5h + 3i and 15hi', '4d+ 2e and 8d+e', 's + s + 2t and 2st', 'f+f+f+g and 3fg', 'b+b+3c and 2b+ 3c', 'p + q + q + r and 2pqr', 'l + m + m and 2lm', '3(x+2) and 3x+2']\",\"question\":\"Which pair of expressions is equivalent for any variable value greater than zero?\",\"src\":\"ori_mmlu-elementary_mathematics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}