{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3c5ad77a-97db-5cdf-8c49-2d9638302ebf","task_key":"default--test--2381","task_revision_id":"1","upstream_id":"2381","short_description":"In order to deal with the legal issue of age discrimination, iris important for…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['only when the employee is over the retirement age', 'whenever an older employee is fired, regardless of the reason', \\\"whenever the employee's age is considered in any employment decision\\\", \\\"whenever the employee's age, instead of work performance, is the basis for firing\\\", 'only when company policy reflects adverse impact on older workers', 'only when the employee has filed a formal complaint about age discrimination', 'only when there are multiple cases of age discrimination in the same company', 'whenever the affected employee can demonstrate substantial economic loss', 'whenever a younger employee is hired to replace an older one', 'only when the majority of employees in a given age group are adversely affected economically']\",\"question\":\"In order to deal with the legal issue of age discrimination, iris important for the psychologist to know that, with regard to the firing of an employee, evidence of age discrimination exists\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}