{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"541281cd-86e7-5271-be7d-7c3b74f5899c","task_key":"default--test--10425","task_revision_id":"1","upstream_id":"10425","short_description":"Which of the following points would Bayesians and frequentists disagree on?","config":"default","split":"test","body":"{\"category\":\"computer science\",\"options\":\"['The use of probabilistic modelling for regression.', 'The use of posterior distributions in a probabilistic model.', 'The use of maximum likelihood estimation in determining parameters.', 'The use of hypothesis testing in statistical inference.', 'The use of empirical Bayes methods in estimating parameters.', 'The use of frequentist methods in estimating population parameters.', 'The use of a non-Gaussian noise model in probabilistic regression.', 'The use of marginal likelihood in model comparison.', 'The use of prior distributions on the parameters in a probabilistic model.', 'The use of class priors in Gaussian Discriminant Analysis.']\",\"question\":\"Which of the following points would Bayesians and frequentists disagree on?\",\"src\":\"ori_mmlu-machine_learning\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}