{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e15fcae4-94bd-55e7-80bd-e210522acbc0","task_key":"default--test--10732","task_revision_id":"1","upstream_id":"10732","short_description":"For a Gaussian Bayes classifier, which one of these structural assumptions is…","config":"default","split":"test","body":"{\"category\":\"computer science\",\"options\":\"['Whether we learn the class centers by Maximum Likelihood or Gradient Descent', 'Whether we assume full class covariance matrices or diagonal class covariance matrices', 'Whether we have equal class priors or priors estimated from the data.', 'Whether we allow classes to have different mean vectors or we force them to share the same mean vector']\",\"question\":\"For a Gaussian Bayes classifier, which one of these structural assumptions is the one that most affects the trade-off between underfitting and overfitting:\",\"src\":\"ori_mmlu-machine_learning\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}