{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"45308ef0-b475-5326-bc2d-8f7618f4aefd","task_key":"default--test--2079","task_revision_id":"1","upstream_id":"2079","short_description":"In personnel selection, it is appropriate to ask a person's age when","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['standardized test results show thar the incumbent does not have all the aptitudes expected for the particular job', 'age is a bona fide occupational requirement', 'properly administered intelligence and personality tests suggest that the incumbent is not likely to perform satisfactorily in the future', \\\"the interviewer is curious about the applicant's age\\\", 'it is for the purposes of company demographics and statistics', 'the incumbent is nearing the typical retirement age', 'the company has a policy of hiring only individuals of a certain age group']\",\"question\":\"In personnel selection, it is appropriate to ask a person's age when\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}