{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b5fea18b-7f1e-5c15-b064-6a6e0212a6b8","task_key":"default--test--10933","task_revision_id":"1","upstream_id":"10933","short_description":"Plato's view is that true beauty is _____.","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['found in everyday objects', 'only found in art', 'not of this world', 'found only in human-made objects', 'only found in the human body', 'a materialistic concept only appreciated by the wealthy', \\\"only found in the mind's understanding\\\", 'nonexistent', 'everywhere in the natural world', 'a subjective concept that differs from person to person']\",\"question\":\"Plato's view is that true beauty is _____.\",\"src\":\"ori_mmlu-philosophy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}