{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"376026fe-fabe-5bc6-845f-de08ca35b9fd","task_key":"default--test--2935","task_revision_id":"1","upstream_id":"2935","short_description":"In humans, a hereditary disorder called xeroderma pigmentosum (XP) can result in…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['Pyrimidine dimer', 'Double strand DNA breaks', 'Deletion of guanine', 'Single strand DNA breaks', 'Addition of uracil', 'Depurination', 'Thymine dimers', 'Methylation of adenine', 'DNA crosslinks', 'Deamination of cytosine']\",\"question\":\"In humans, a hereditary disorder called xeroderma pigmentosum (XP) can result in the inability to repair ultraviolet damage to DNA. Which of the following molecular lesions is most likely to accrue in individuals with XP?\",\"src\":\"ori_mmlu-college_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}