{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1d0fd988-89e0-5ee4-9aab-9300821e978a","task_key":"default--test--7184","task_revision_id":"1","upstream_id":"7184","short_description":"Which one of the following statements is true concerning VARs?","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['The coefficient estimates in VARs are always positive', 'VARs require all variables to be of the same sign for effective forecasting', 'The coefficient estimates have intuitive theoretical interpretations', 'VARs often produce better forecasts than simultaneous equation structural models', 'All of the components of a VAR must be stationary before it can be used for forecasting', 'The coefficient estimates have no theoretical interpretations in VARs', 'VARs cannot be used for forecasting purposes', 'All components of a VAR must be non-stationary for accurate forecasting', 'VARs produce poorer forecasts when compared to standalone variable models']\",\"question\":\"Which one of the following statements is true concerning VARs?\",\"src\":\"ori_mmlu-econometrics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}