{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"66a8010c-dd15-5489-9d24-db160614c46c","task_key":"default--test--8779","task_revision_id":"1","upstream_id":"8779","short_description":"Which of the following is a false statement about simple random samples?","config":"default","split":"test","body":"{\"category\":\"math\",\"options\":\"['The size of the population affects the randomness of the sample.', 'Every element of the population has an equal chance of being picked.', 'A sample must be reasonably large to be properly considered a simple random sample.', 'Attributes of a simple random sample may be very different from attributes of the population.', 'Inspection of a sample will give no indication of whether or not it is a simple random sample.']\",\"question\":\"Which of the following is a false statement about simple random samples?\",\"src\":\"ori_mmlu-high_school_statistics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}