{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"19c83dde-e068-5aa6-8cb9-a3f6e3e2d697","task_key":"default--test--6287","task_revision_id":"1","upstream_id":"6287","short_description":"One reason that older adults show high levels of optimism is that","config":"default","split":"test","body":"{\"category\":\"health\",\"options\":\"['Our measures have low validity', 'Many pessimists die at a younger age', 'Their cognitive abilities decline with age', 'They do not really know what is going on', 'They tend to focus on the past rather than the future', 'This may be a warning sign for dementia']\",\"question\":\"One reason that older adults show high levels of optimism is that\",\"src\":\"ori_mmlu-human_aging\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}