{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a207efd6-d86d-59c0-a712-9a9903c53947","task_key":"default--test--11262","task_revision_id":"1","upstream_id":"11262","short_description":"What does SCNT stand for, as used in the discussion on cloning?","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['somatic cellular nuclear transmission', 'somatic cell nuclear transfer', 'none of the above', 'specialized cell nuclear transfer', 'skin cell nuclear transition', 'somatic cell nucleotide transition', 'skin cell nuclear transfer', 'stem cell nuclear transmission', 'stem cell nuclear transformation', 'synthetic cell nuclear transfer']\",\"question\":\" What does SCNT stand for, as used in the discussion on cloning?\",\"src\":\"ori_mmlu-moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}