{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d6a3473e-1ab5-5f79-bd64-be8dfc9ac00d","task_key":"default--test--3206","task_revision_id":"1","upstream_id":"3206","short_description":"The geneticist Mary Lyon hypothesized the existence of structures visible just…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['In the early female embryo, one copy of the X chromosome becomes inactivated in every body cell.', 'The same chromosome in every cell of a normal female is inactivated.', 'A male with the XXY genotype will have one Barr body.', 'Barr bodies consist of highly methylated DNA.']\",\"question\":\"The geneticist Mary Lyon hypothesized the existence of structures visible just under the nuclear membrane in mammals, which were later named Barr bodies. Which of the following statement is NOT correct about Barr bodies?\",\"src\":\"ori_mmlu-high_school_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}