{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e74963fd-25a4-5c45-90ff-1a16e4e99694","task_key":"default--test--2091","task_revision_id":"1","upstream_id":"2091","short_description":"In general, it can be said that job satisfaction","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"[\\\"has no relation with the individual's personal life\\\", 'tends to decrease steadily as tenure on a job increases', 'is always high in jobs that require high levels of education', \\\"is solely dependent on the job's financial compensation\\\", 'is only achieved if an individual is in a management position', 'only improves when the individual has a high level of job security', 'is not influenced by the working environment', 'is higher in groups that do not have to assume responsibility', 'seems to increase considerably with an increase in wages', 'is not highly correlated with job performance']\",\"question\":\"In general, it can be said that job satisfaction\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}