{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3fe3032d-388d-5423-bd19-5c65c6f27233","task_key":"default--test--2886","task_revision_id":"1","upstream_id":"2886","short_description":"A gal^+ / gal^- cell is produced by an abortive transduction. Will it grow in a…","config":"default","split":"test","body":"{\"category\":\"biology\",\"options\":\"['The colony does not grow at all.', 'The resulting colony is minute.', 'The colony grows slowly, reaching average size eventually.', 'The colony grows but is unable to utilize galactose efficiently.', 'The colony grows to an unusually large size before ceasing growth.', 'The colony grows initially but then stops abruptly.', 'The resulting colony is large.', 'The colony exhibits fluctuating growth rates.', 'The colony grows rapidly, surpassing average size.', 'The resulting colony is average in size.']\",\"question\":\"A gal^+ / gal^- cell is produced by an abortive transduction. Will it grow in a medium in whichgalactoseis the sole carbon source?\",\"src\":\"stemez-Genetics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}