{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c010d32a-f87a-5da9-9b32-d82a2b3652b1","task_key":"default--test--6909","task_revision_id":"1","upstream_id":"6909","short_description":"Under which of the following situations would bootstrapping be preferred to pure…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['(i) and (ii) only', '(ii) and (iii) only', '(i) and (iii) only', '(i) and (iv) only', '(ii) and (iv) only', '(iii) and (iv) only', '(i), (ii), (iii), and (iv)', '(i), (iii), and (iv) only', '(ii), (iii), and (iv) only', '(i), (ii), and (iv) only']\",\"question\":\"Under which of the following situations would bootstrapping be preferred to pure simulation?\\n\\ni) If it is desired that the distributional properties of the data in the experiment\\n\\nare the same as those of some actual data\\n\\n\\nii) If it is desired that the distributional properties of the data in the experiment\\n\\nare known exactly\\n\\n\\niii) If the distributional properties of the actual data are unknown\\n\\n\\niv) If the sample of actual data available is very small\",\"src\":\"ori_mmlu-econometrics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}