{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d292e106-3b08-5b04-a6ea-f9f08ebd876a","task_key":"default--test--6200","task_revision_id":"1","upstream_id":"6200","short_description":"Which membrane fatty acids are precursors for eicosanoids?","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['C18 and C20 saturated fatty acids', 'C20 and C22 saturated fatty acids', 'C16 and C18 polyunsaturated fatty acids', 'C16 and C22 polyunsaturated fatty acids', 'C18 and C22 polyunsaturated fatty acids', 'C18 and C22 monounsaturated fatty acids', 'C20 and C22 monounsaturated fatty acids', 'C16 and C20 saturated fatty acids', 'C20 and C22 polyunsaturated fatty acids', 'C18 and C20 monounsaturated fatty acids']\",\"question\":\"Which membrane fatty acids are precursors for eicosanoids?\\n\",\"src\":\"ori_mmlu-nutrition\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}