{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9326d483-75c9-52a6-8b55-0de1c4ce1d22","task_key":"default--test--2812","task_revision_id":"1","upstream_id":"2812","short_description":"If two modern organisms are distantly related in an evolutionary sense, then one…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['their rate of mutation should be identical.', 'they should have the exact same DNA structure.', 'they should have identical physical characteristics.', 'they should share the same number of chromosomes.', 'they live in very different habitats.', 'their chromosomes should be very similar.', 'they shared a common ancestor relatively recently.', 'they should share fewer homologous structures than two more closely related organisms.', 'they should share more homologous structures than two more closely related organisms.', 'they both evolve at the same rate.']\",\"question\":\"If two modern organisms are distantly related in an evolutionary sense, then one should expect that\",\"src\":\"ori_mmlu-high_school_biology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}