{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"490cf407-8fad-5e84-b5cf-b7c0fcd948f3","task_key":"default--test--830","task_revision_id":"1","upstream_id":"830","short_description":"By which principle is utilitarianism defined?","config":"default","split":"test","body":"{\"category\":\"business\",\"options\":\"['Maximising personal pleasure', 'Maximisation of outcomes for individuals', 'The least worst outcome for everybody', 'Equality of opportunity for everyone', 'Equality for everyone', 'The best outcome for the minority of people', 'Ensuring no harm is done to any individual', 'The best outcome for the majority of people', 'The least worst outcome for the majority of people', 'Maximising wealth and resources for everyone']\",\"question\":\"By which principle is utilitarianism defined?\",\"src\":\"ori_mmlu-management\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}