{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6778465a-110b-50ba-8970-8d1e80ba03e0","task_key":"default--test--434","task_revision_id":"1","upstream_id":"434","short_description":"Pick the correct description of the following term: Utilitarianism is…","config":"default","split":"test","body":"{\"category\":\"business\",\"options\":\"['A theory which states that an action is morally right if it promotes the greatest good for the decision maker', 'A theory which states that an action is morally right if it results in a balanced distribution of good and bad outcomes', 'A theory which states that an action is morally right if it promotes the greatest good for the fewest people', 'A theory which states that an action is morally right if it promotes the least harm for the greatest number', 'A theory which states that an action is morally wrong if it promotes the greatest good for the majority', 'A theory which states that an action is morally incorrect if it promotes the greatest good for the greatest number', 'A theory which states that an action is morally right if it promotes the greatest good for the greatest number', 'A theory which states that an action is morally right if it promotes the greatest harm for the smallest number', 'A theory which states that an action is morally right if it promotes the greatest good for the individual performing the action', 'A theory which states that an action is morally right if it promotes the greatest good for the smallest number']\",\"question\":\"Pick the correct description of the following term: Utilitarianism is…\",\"src\":\"ori_mmlu-business_ethics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}