{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3c893dca-e17c-5e2a-b9ce-2d3a41ef03aa","task_key":"default--test--7448","task_revision_id":"1","upstream_id":"7448","short_description":"Which of the following are plausible approaches to dealing with a model that…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['(iii) and (iv) only', '(i), (iii), and (iv) only', '(i), (ii), and (iv) only', '(ii), (iii), and (iv) only', '(i) and (ii) only', '(i) and (iv) only', '(i) and (iii) only', '(ii) and (iv) only', '(i), (ii), (iii), and (iv)', '(i), (ii), and (iii) only']\",\"question\":\"Which of the following are plausible approaches to dealing with a model that exhibits heteroscedasticity?\\n\\ni) Take logarithms of each of the variables\\n\\nii) Use suitably modified standard errors\\n\\niii) Use a generalised least squares procedure\\n\\niv) Add lagged values of the variables to the regression equation.\",\"src\":\"ori_mmlu-econometrics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}