{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"29f4764a-24f0-5fda-bbf8-4e669305ba1a","task_key":"default--test--11064","task_revision_id":"1","upstream_id":"11064","short_description":"Plato's suggestion that knowledge is innate or remembered as a result of being…","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['how anyone can recognize the correct answer to a question without already knowing the answer.', 'how the soul can reincarnate if it carries no memory of past lives.', 'how knowledge of the Forms can ever be anything other than a generalization of experience.', 'how a person can remember anything about the realm of the Forms after the shock of being born into this world.', 'how concepts bound to the realm of becoming have meaning only when associated with the realm of Being.', 'how a person can possess knowledge without having been taught.', 'how a person can forget something they once knew clearly.', 'how it is possible to know something that one has not experienced.', 'how knowledge can be both innate and learned at the same time.', 'how the human mind can conceive of the Forms without physical representation.']\",\"question\":\"Plato's suggestion that knowledge is innate or remembered as a result of being triggered by experience is in response to a paradox he sets up for himself.  The paradox, now referred to as Meno's Paradox, has to do with the question of:\",\"src\":\"ori_mmlu-philosophy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}