{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"76423e3a-9944-57b6-a092-8f9c562f78b2","task_key":"default--test--7371","task_revision_id":"1","upstream_id":"7371","short_description":"Which of the following statements are true concerning a triangular or recursive…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['(i), (ii), (iii), and (iv)', '(iii) and (iv) only', '(i), (ii), and (iii) only', '(i) and (iii) only', '(i), (iii), and (iv) only', '(ii) only', '(ii) and (iii) only', '(i) and (iv) only', '(i) and (ii) only', '(ii) and (iv) only']\",\"question\":\"Which of the following statements are true concerning a triangular or recursive system?\\n\\ni) The parameters can be validly estimated using separate applications of OLS to\\n\\neach equation\\n\\n\\nii) The independent variables may be correlated with the error terms in other\\n\\nequations\\n\\n\\niii) An application of 2SLS would lead to unbiased but inefficient parameter estimates\\n\\n\\niv) The independent variables may be correlated with the error terms in the equations\\n\\nin which they appear as independent variables\",\"src\":\"ori_mmlu-econometrics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}