{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fb3a9e9f-3f46-552c-815a-db38a68b0237","task_key":"all--test--fb3a9e9f-3f46-552c-815a-db38a68b0237","task_revision_id":"1","upstream_id":"","short_description":"Under which of the following situations would bootstrapping be preferred to pure…","config":"all","split":"test","body":"{\"choices\":\"['(ii) and (iv) only', '(i) and (iii) only', '(i), (ii), and (iv) only', '(i), (ii), (iii), and (iv)']\",\"question\":\"Under which of the following situations would bootstrapping be preferred to pure simulation?\\n\\ni) If it is desired that the distributional properties of the data in the experiment\\n\\nare the same as those of some actual data\\n\\n\\nii) If it is desired that the distributional properties of the data in the experiment\\n\\nare known exactly\\n\\n\\niii) If the distributional properties of the actual data are unknown\\n\\n\\niv) If the sample of actual data available is very small\",\"subject\":\"econometrics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}