{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9fb73f64-4475-57b1-9af4-61b5807e9e6d","task_key":"default--test--2706","task_revision_id":"1","upstream_id":"2706","short_description":"A senior psychologist hires a junior psychologist to conduct a research study.…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['listed as a co-author with the senior psychologist', 'promoted to a senior position', 'given a percentage of any profits from the research', 'given credit in a footnote', 'credited as the editor of the research', 'listed in the acknowledgements section', 'thanked for doing a good job', 'listed as the first author', 'given a bonus or raise', 'listed as an author']\",\"question\":\"A senior psychologist hires a junior psychologist to conduct a research study. The junior psychologist “runs the experiment” and helps analyze and write up the results. When the research is submitted for publication, professional ethics require thar the junior psychologist be\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}