{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"82d9cf2e-42a2-5100-845c-0d749db60dde","task_key":"default--test--2202","task_revision_id":"1","upstream_id":"2202","short_description":"The function of the correction for attenuation is to","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['adjust the correlation coefficient for any potential influence of a third variable', 'reduce the correlation between two variables by accounting for measurement errors', 'provide a method for correcting bias in the distribution of scores', 'provide an estimate of the predictive validity of a test-thar is independent of chance factors in any specific validation sample', 'estimate the correlation between more than two variables if one or more could be measured without error', 'show how adding or subtracting items affects the reliability of a test', 'estimate the correlation between wo variables if one or both could be measured without error', 'correct an obtained correlation coefficient for restriction of range in either variable', 'adjust the value of the correlation coefficient to account for sample size', 'provide a means to increase the reliability of a test by eliminating outliers']\",\"question\":\"The function of the correction for attenuation is to\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}