{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c5b0d875-d577-59b4-b703-810855ce5e39","task_key":"default--test--7247","task_revision_id":"1","upstream_id":"7247","short_description":"Which of the following statements concerning the regression population and…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['The population is the total collection of all items of interest', 'The population can be infinite', 'In theory, the sample could be larger than the population', 'A random sample is one where each individual item from the population is equally likely to be drawn.']\",\"question\":\"Which of the following statements concerning the regression population and sample is FALSE?\",\"src\":\"ori_mmlu-econometrics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}