{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"31bda8fc-8656-5726-81b0-9f25422d981e","task_key":"default--test--6514","task_revision_id":"1","upstream_id":"6514","short_description":"The stomach lining contains several types of cells. Which statement about them…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Parietal cells secrete pepsinogen, the inactive precursor of pepsin', 'G cells secrete the enzyme amylase', 'Chief cells the hormone gastrin (which stimulates acid secretion).', 'Mucous cells secrete the hormone secretin', 'Enterochromaffin-like cells secrete the enzyme lipase', 'Parietal cells secrete the hormone ghrelin', 'Mucous cells secrete the glycoprotein mucin', 'Chief cells secrete bile acid', 'D cells secrete the hormone cholecystokinin']\",\"question\":\"The stomach lining contains several types of cells. Which statement about them is correct?\\n\",\"src\":\"ori_mmlu-nutrition\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}