{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0f560a89-2523-561d-b05f-1d65eee35759","task_key":"default--test--2618","task_revision_id":"1","upstream_id":"2618","short_description":"The primary advantage of the forced-choice distribution of rated attributes in…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['allows for more flexibility in rating employees', 'reduces the bias in ratings as it is based on direct observation', 'is less time-consuming, since it does not require in-depth analysis', 'is more accurate as it focuses on measurable attributes', 'is more objective, since it does not deal with the strengths and weaknesses of the employee', 'enhances the reliability of ratings', 'eliminates statistical concerns about ipsative measurement', 'is easier and faster to use, since only two choices are available', 'eliminates the need for continuous feedback and follow-up', 'is more comprehensive as it covers all aspects of performance']\",\"question\":\"The primary advantage of the forced-choice distribution of rated attributes in performance appraisal is that it\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}