{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"44b4f20f-0e65-5230-8fda-5249b3487dd5","task_key":"default--test--6137","task_revision_id":"1","upstream_id":"6137","short_description":"When branched chain amino acids are deaminated in muscle, the ammonia produced…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['converted into citrulline and released from the muscle.', 'converted into urea and released from the muscle.', 'used to synthesise purines and pyrimidines in the muscle.', 'converted into alanine and glutamine and released from the muscle.', 'converted into glycine and released from the muscle.', 'converted into arginine and released from the muscle.', 'converted into creatine and released from the muscle.', 'used to synthesise fatty acids in the muscle.', 'converted into aspartic acid and released from the muscle.', 'used to synthesise proteins in the muscle.']\",\"question\":\"When branched chain amino acids are deaminated in muscle, the ammonia produced is mostly:\",\"src\":\"ori_mmlu-college_medicine\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}