{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c8b24006-7ce5-5a33-b828-53e2833d0346","task_key":"default--test--10633","task_revision_id":"1","upstream_id":"10633","short_description":"When doing least-squares regression with regularisation (assuming that the…","config":"default","split":"test","body":"{\"category\":\"computer science\",\"options\":\"['will not have any effect on the testing error.', 'will always increase the testing error.', 'may decrease or increase the testing error.', 'will never increase the training error.', 'will always decrease the testing error.', 'may decrease or increase the training error.', 'will not have any effect on the training error.', 'will never decrease the testing error.', 'will never decrease the training error.', 'will never increase']\",\"question\":\"When doing least-squares regression with regularisation (assuming that the optimisation can be done exactly), increasing the value of the regularisation parameter λ the testing error.\",\"src\":\"ori_mmlu-machine_learning\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}