{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c29d0a1a-d903-5dbb-bbd4-0c42231e7b7d","task_key":"default--test--11029","task_revision_id":"1","upstream_id":"11029","short_description":"Epicurus conceives of death as:","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['the greatest harm that one can suffer.', 'an illusion that does not truly exist.', 'the transformation into a higher being.', 'one component of the continuous cycle of rebirth.', 'a state of eternal suffering.', 'the ultimate state of bliss.', 'the deprivation of all sensation.', \\\"the final stage of life's journey.\\\", 'the beginning of the afterlife.', 'a necessary stage to reach enlightenment.']\",\"question\":\"Epicurus conceives of death as:\",\"src\":\"ori_mmlu-philosophy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}