{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f52622f1-a8f4-53af-a275-c897e35b4390","task_key":"default--test--10451","task_revision_id":"1","upstream_id":"10451","short_description":"Statement 1| Linear regression estimator has the smallest variance among all…","config":"default","split":"test","body":"{\"category\":\"computer science\",\"options\":\"['The first statement is sometimes true, False', 'True, The second statement is sometimes true', 'False, False', 'False, True', 'Both statements are sometimes true', 'Both statements are partially true', 'False, The second statement is sometimes false', 'The first statement is sometimes false, True', 'True, True', 'True, False']\",\"question\":\"Statement 1| Linear regression estimator has the smallest variance among all unbiased estimators. Statement 2| The coefficients α assigned to the classifiers assembled by AdaBoost are always non-negative.\",\"src\":\"ori_mmlu-machine_learning\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}