{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8eafae5b-f1ab-50e2-9551-96fd2bc90b82","task_key":"default--test--10558","task_revision_id":"1","upstream_id":"10558","short_description":"Statement 1| The maximum margin decision boundaries that support vector machines…","config":"default","split":"test","body":"{\"category\":\"computer science\",\"options\":\"['True, True only for 2D data', 'False, True but requires specific tuning', 'True, False', 'False, True only with RBF kernel', 'True, False only for high dimensional data', 'False, True', 'True, True', 'False, True only for linearly separable data', 'True, False unless data is normalized', 'False, False']\",\"question\":\"Statement 1| The maximum margin decision boundaries that support vector machines construct have the lowest generalization error among all linear classifiers. Statement 2| Any decision boundary that we get from a generative model with classconditional Gaussian distributions could in principle be reproduced with an SVM and a polynomial kernel of degree less than or equal to three.\",\"src\":\"ori_mmlu-machine_learning\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}