{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9ad55cb2-1b55-548f-8665-45f56841b9cc","task_key":"default--test--10525","task_revision_id":"1","upstream_id":"10525","short_description":"Statement 1| Density estimation (using say, the kernel density estimator) can be…","config":"default","split":"test","body":"{\"category\":\"computer science\",\"options\":\"['Partially True, Partially False', 'Mostly True, Mostly False', 'Partially False, Partially True', 'False for certain cases, True for certain cases', 'True, True', 'True, False', 'False, False', 'False, True', 'Mostly False, Mostly True', 'True for certain cases, False for certain cases']\",\"question\":\"Statement 1| Density estimation (using say, the kernel density estimator) can be used to perform classification. Statement 2| The correspondence between logistic regression and Gaussian Naive Bayes (with identity class covariances) means that there is a one-to-one correspondence between the parameters of the two classifiers.\",\"src\":\"ori_mmlu-machine_learning\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}