{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4131b3f7-9b76-52fd-bc6a-4d392c651371","task_key":"default--test--10935","task_revision_id":"1","upstream_id":"10935","short_description":"\"There is no rationale for myth because it is through myth that reason itself is…","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['metaphysicians are justified in reasoning as they do because there is only one true answer about being.', 'mythos is ultimately based on logos, just as myth is ultimately based on reasoning or thinking.', 'myth and reason are interchangeable terms, each defining the other depending on the situation.', 'myth does not \\\"explain\\\" how things are related as much as it simply reveals them as related.', 'myth and reason are completely separate entities with no relation to each other.', 'myth is a form of reason because it helps to explain the unknown.', 'myths can be explained by reason, but reason cannot be explained by myths.', 'myth and reason are the same: \\\"myth\\\" defines \\\"reason,\\\" and \\\"reason\\\" defines \\\"myth.\\\"']\",\"question\":\"\\\"There is no rationale for myth because it is through myth that reason itself is defined.\\\"  This means that:\",\"src\":\"ori_mmlu-philosophy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}