{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ed5f445e-1b5c-5eeb-b5db-9d56418c4b60","task_key":"default--test--2872","task_revision_id":"1","upstream_id":"2872","short_description":"The polytene chromosomes seen in Drosophila silvarentis with the aid of a light…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['multiple copies of a single chromosome', 'not visible even under a light microscope', 'produced only during mitosis', 'produced by repeated rounds of nuclear division without DNA replication', 'single copies of multiple chromosomes', 'not actively carrying out transcription', 'produced by single round of DNA replication without nuclear division', 'produced only during meiosis', 'produced by repeated rounds of DNA replication followed by nuclear division']\",\"question\":\"The polytene chromosomes seen in Drosophila silvarentis with the aid of a light microscope are\",\"src\":\"ori_mmlu-college_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}