{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9728df7e-d60e-59fe-aba5-2023fee0a022","task_key":"default--test--2633","task_revision_id":"1","upstream_id":"2633","short_description":"In personnel selection, the selection ratio is defined as the","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['ratio of successful co unsuccessful workers', 'ratio of the number of successful selected workers to the difficulty level of the job', 'percent of workers correctly placed in the group hired to do the job', 'ratio of the number of workers needed to the number of applicants available']\",\"question\":\"In personnel selection, the selection ratio is defined as the\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}