{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"322fd357-870e-5f9f-9b1b-286675c04355","task_key":"default--test--6911","task_revision_id":"1","upstream_id":"6911","short_description":"Which of the following statements is true concerning the population regression…","config":"default","split":"test","body":"{\"category\":\"economics\",\"options\":\"['The PRF and SRF are interchangeable terms referring to the same concept.', 'The PRF is the model used for prediction, while the SRF is used for explanation', 'The PRF is used to infer likely values of the SRF', 'Whether the model is good can be determined by comparing the SRF and the PRF', 'The SRF is a theoretical construct, while the PRF is an empirical construct', 'The PRF is a description of the process thought to be generating the data.', 'The SRF is a description of the process thought to be generating the data', 'The PRF is the estimated model']\",\"question\":\"Which of the following statements is true concerning the population regression function (PRF) and sample regression function (SRF)?\",\"src\":\"ori_mmlu-econometrics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}