{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"12234963-a0b0-5c08-ae5a-ffc6c38b06f7","task_key":"default--test--7659","task_revision_id":"1","upstream_id":"7659","short_description":"Which of the following statements are correct concerning the use of antithetic…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['(i), (iii), and (iv) only', '(ii) and (iii) only', '(iii) and (iv) only', '(i) and (iv) only', '(ii), (iii), and (iv) only', '(i), (ii), and (iv) only', '(i), (ii), (iii), and (iv)', '(i) and (iii) only', '(i), (ii), and (iii) only', '(ii) and (iv) only']\",\"question\":\"Which of the following statements are correct concerning the use of antithetic variates as part of a Monte Carlo experiment?\\n\\ni) Antithetic variates work by reducing the number of replications required to cover the whole probability space\\n\\nii) Antithetic variates involve employing a similar variable to that used in the simulation, but whose properties are known analytically\\n\\niii) Antithetic variates involve using the negative of each of the random draws and repeating the experiment using those values as the draws\\n\\niv) Antithetic variates involve taking one over each of the random draws and repeating the experiment using those values as the draws\",\"src\":\"ori_mmlu-econometrics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}