{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"75fe613e-ab62-5d8c-bf2f-434b8fd6ac9a","task_key":"default--test--8014","task_revision_id":"1","upstream_id":"8014","short_description":"A researcher is hoping to find a predictive linear relationship between the…","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['I only', 'I, II, and III', 'None of the above', 'I and II only', 'III only', 'II and III only', 'I and III only', 'Only II is likely, while I and III are not', 'II only', 'I and II are likely, but III is not']\",\"question\":\"A researcher is hoping to find a predictive linear relationship between the explanatory and response variables in her study. Accordingly, as part of her analysis she plans to generate a 95% confidence interval for the slope of the regression line for the two variables. The interval is determined to be (0.45, 0.80). Which of the following is (are) true? (Assume conditions for inference are met.) I. She has good evidence of a linear relationship between the variables. II. It is likely that there is a non-zero correlation (r) between the two variables. III. It is likely that the true slope of the regression line is 0.\",\"src\":\"ori_mmlu-high_school_statistics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}