{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7c0cf04a-3084-5c72-b3d1-5d7b598bafac","task_key":"default--test--6289","task_revision_id":"1","upstream_id":"6289","short_description":"Which of the following could very well be a cohort difference rather than a…","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Loss of muscle mass', 'Shrinkage of the thymus gland', 'Differences in weight', 'Changes in metabolism rate', 'Changes in eyesight', 'Enlargement of the heart', 'Increase in wrinkles', 'Loss of hair color', 'Decreasing bone density', 'Increasing size of nose and ears']\",\"question\":\"Which of the following could very well be a cohort difference rather than a change that occurs with age?\",\"src\":\"ori_mmlu-human_aging\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}