{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"58005283-8e69-5d1b-9c90-7ad1efeeb416","task_key":"default--test--3155","task_revision_id":"1","upstream_id":"3155","short_description":"Which of the changes below following the start codon in an mRNA would most…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['a deletion of a nucleotide triplet', 'a deletion of a nucleotide doublet', 'a multiple nucleotide substitution in the first, second, and third codon positions', 'a single nucleotide insertion after the first codon position', 'a single nucleotide substitution of the nucleotide occupying the first codon position', 'a double nucleotide substitution in the first and second codon positions', 'a deletion of a single nucleotide', 'a single nucleotide substitution of the nucleotide occupying the third codon position', 'a deletion of a quadruplet of nucleotides', 'a single nucleotide substitution of the nucleotide occupying the second codon position']\",\"question\":\"Which of the changes below following the start codon in an mRNA would most likely have the greatest deleterious effect?\",\"src\":\"ori_mmlu-high_school_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}