{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3a72137c-2f31-5ec7-8c10-37676b2b4095","task_key":"default--test--807","task_revision_id":"1","upstream_id":"807","short_description":"The marketing research firm of Burrows, Heller and Larimer wants to estimate the…","config":"default","split":"test","body":"{\"category\":\"business\",\"options\":\"['The confidence interval includes zero, indicating no difference in proportions', 'The proportions are equal', 'The proportions are not equal', 'The confidence interval does not include zero, indicating a significant difference in proportions', 'The proportion of women is higher', 'The confidence interval is too wide to determine a significant difference', 'The proportion of men is higher', 'No conclusion can be drawn without further data', 'The proportion of men is significantly lower', 'The proportion of women is significantly lower']\",\"question\":\"The marketing research firm of Burrows, Heller and Larimer wants to estimate the proportions of men and women who are familiar with a shoe polish. In a sample (random) of 100 men and 200 women it is found that 20 men and 60 women had used this particular shoe polish. Compute a 95% confidence interval for the difference in pro portions between men and women familiar with the product. Use this to test the hypothesis that the proportions are equal.\",\"src\":\"stemez-Business\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}