{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"719ee056-7606-54d7-b24d-5d0cbbfe1ae6","task_key":"default--test--7910","task_revision_id":"1","upstream_id":"7910","short_description":"Given that the sample has a standard deviation of zero, which of the following…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['The population mean is also zero.', 'The sample mean and sample mode are not equal.', 'The sample size must be very small.', 'All data points in the sample are outliers.', 'The sample may have outliers.', 'The sample has a skewed distribution.', 'The standard deviation of the population is also zero.', 'The population has a symmetric distribution.', 'The sample mean and sample median are equal.']\",\"question\":\"Given that the sample has a standard deviation of zero, which of the following is a true statement?\",\"src\":\"ori_mmlu-high_school_statistics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}