{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a40594e5-bfb0-58b9-b2c0-609d3b742214","task_key":"default--test--6814","task_revision_id":"1","upstream_id":"6814","short_description":"Which are the FIVE main series of apoproteins that have been identified?","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['apoA, apo(a), apoB, apoC and apoG', 'apoA, apoB, apoC, apo E, and apoL', 'apoA, apoB, apoC, apoD, and apo(a)', 'apoA, apoB, apoC, apoD, and apoE', 'apo(a), apoB, apoC, apoD, and apoE', 'apoB, apoC, apoD, apoE and apoM', 'apoA, apoB, apoC, apoD, and apoF', 'apoA, apo(a), apoB, apoC and apoE', 'apoA, apo(a), apoB, apoC and apoH', 'apoA, apoB, apoC, apo(a) and apoM']\",\"question\":\"Which are the FIVE main series of apoproteins that have been identified?\\n \",\"src\":\"ori_mmlu-nutrition\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}