{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a06089dc-614d-5a07-9fbf-f5b48957128b","task_key":"default--test--10454","task_revision_id":"1","upstream_id":"10454","short_description":"Statement 1| Since the VC dimension for an SVM with a Radial Base Kernel is…","config":"default","split":"test","body":"{\"category\":\"computer science\",\"options\":\"['False, False', 'False, True', 'Neither True nor False, False', 'Neither True nor False, True', 'True, True', 'False, Neither True nor False', 'Neither True nor False, Neither True nor False', 'True, False', 'True, Neither True nor False', 'True, False and False, True']\",\"question\":\"Statement 1| Since the VC dimension for an SVM with a Radial Base Kernel is infinite, such an SVM must be worse than an SVM with polynomial kernel which has a finite VC dimension. Statement 2| A two layer neural network with linear activation functions is essentially a weighted combination of linear separators, trained on a given dataset; the boosting algorithm built on linear separators also finds a combination of linear separators, therefore these two algorithms will give the same result.\",\"src\":\"ori_mmlu-machine_learning\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}