{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"aab03791-2947-5780-b391-88a9ec71d335","task_key":"default--test--8338","task_revision_id":"1","upstream_id":"8338","short_description":"Fifty oranges of one variety were selected at random and weighed. A 95 percent…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['In repeated samplings of the same size, 95 percent of the intervals constructed will contain the true mean weight of the oranges of this variety.', 'Ninety-five percent of the oranges of this variety weigh between 7.58 and 8.72 ounces.', 'In repeated samplings of the same size, 5 percent of the intervals constructed will contain the true mean weight of the oranges of this variety.', 'Ninety-five percent of the oranges in this sample weigh exactly 7.58 or 8.72 ounces.', 'In repeated samplings of the same size, 95 percent of the intervals constructed will contain the sampled mean weight of the oranges.', 'In repeated samplings of the same size, 95 percent of the oranges will weigh between 7.58 and 8.72 ounces.', 'In repeated samplings of the same size, 95 percent of the intervals constructed will exclude the true mean weight of the oranges of this variety.', 'Ninety-five percent of the oranges of this variety weigh exactly 7.58 or 8.72 ounces.', 'The interval (7.58, 8.72) ounces represents the range of all possible mean weights of the oranges of this variety.', 'Ninety-five percent of the oranges in this sample weigh between 7.58 and 8.72 ounces.']\",\"question\":\"Fifty oranges of one variety were selected at random and weighed. A 95 percent confidence interval computed from the sample weights to estimate the mean weight of oranges of this variety is (7.58, 8.72) ounces. Which of the following statements is true with respect to the confidence level used?\",\"src\":\"ori_mmlu-high_school_statistics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}