{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f355408a-eb07-574e-80d2-e344a9de822a","task_key":"default--test--10528","task_revision_id":"1","upstream_id":"10528","short_description":"Statement 1| PCA and Spectral Clustering (such as Andrew Ng’s) perform…","config":"default","split":"test","body":"{\"category\":\"computer science\",\"options\":\"['Neither true nor false, Neither true nor false', 'False, Neither true nor false', 'True, False, Neither true nor false', 'False, True', 'True, Neither true nor false', 'Neither true nor false, True', 'True, True', 'False, False', 'True, False', 'Neither true nor false, False']\",\"question\":\"Statement 1| PCA and Spectral Clustering (such as Andrew Ng’s) perform eigendecomposition on two different matrices. However, the size of these two matrices are the same. Statement 2| Since classification is a special case of regression, logistic regression is a special case of linear regression.\",\"src\":\"ori_mmlu-machine_learning\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}