{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"63b3bc6f-344b-536e-a341-2fbdfcf0a0b5","task_key":"default--test--11067","task_revision_id":"1","upstream_id":"11067","short_description":"In Hobbes’s view, to say something is good is to say that:","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['it aligns with natural law.', 'it benefits society as a whole.', 'it is conducive to human happiness.', 'most people ought to desire it.', 'it is universally desired.', 'you desire it.', 'most people desire it.', 'it is desired by powerful individuals.', 'it is morally right.', 'it leads to a state of peace.']\",\"question\":\"In Hobbes’s view, to say something is good is to say that:\",\"src\":\"ori_mmlu-philosophy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}