{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"78548c13-c503-581a-acb4-237f2cc9ad21","task_key":"default--test--1490","task_revision_id":"1","upstream_id":"1490","short_description":"Why is it important to separate the concept of punishment from its…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Because the concept of punishment has evolved over time.', 'Because punishment can be justified in multiple ways.', 'Because any definition of punishment should be value-neutral.', 'Because the practice of punishment is separate from its justification.', 'Because the justification of punishment varies across cultures.', 'Because the concept of punishment is often misunderstood.', 'Because the concept of punishment is controversial.', 'Because the justification of punishment is a matter of personal opinion.', 'Because its justification depends on the concept employed.']\",\"question\":\"Why is it important to separate the concept of punishment from its justification?\",\"src\":\"ori_mmlu-jurisprudence\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}