{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"17231d8e-353d-5661-b5c2-7f39441dcfa7","task_key":"default--test--11256","task_revision_id":"1","upstream_id":"11256","short_description":"Ross claims that we learn of our prima facie duties:","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['by observing the consequences of our actions.', 'by apprehending general principles, and then inferring the prima facie rightness of particular acts.', 'by intuitively understanding moral obligations.', 'through legal regulations and laws.', 'by seeing the prima facie rightness of particular acts, and then apprehending general principles.', 'through societal norms and cultural values.', 'from the moral judgments we make in various situations.', 'from the explicit moral instruction we receive as children.', 'by proving them philosophically.', 'from religious teachings or scriptures.']\",\"question\":\"Ross claims that we learn of our prima facie duties:\",\"src\":\"ori_mmlu-philosophy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}