{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"04c346b8-027a-5093-b137-a2c7f1caca50","task_key":"all--test--04c346b8-027a-5093-b137-a2c7f1caca50","task_revision_id":"1","upstream_id":"","short_description":"Which of the following statements are correct concerning the use of antithetic…","config":"all","split":"test","body":"{\"choices\":\"['(ii) and (iv) only', '(i) and (iii) only', '(i), (ii), and (iv) only', '(i), (ii), (iii), and (iv)']\",\"question\":\"Which of the following statements are correct concerning the use of antithetic variates as part of a Monte Carlo experiment?\\n\\ni) Antithetic variates work by reducing the number of replications required to cover the whole probability space\\n\\nii) Antithetic variates involve employing a similar variable to that used in the simulation, but whose properties are known analytically\\n\\niii) Antithetic variates involve using the negative of each of the random draws and repeating the experiment using those values as the draws\\n\\niv) Antithetic variates involve taking one over each of the random draws and repeating the experiment using those values as the draws\",\"subject\":\"econometrics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}