{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f99661c7-e699-565e-adb5-3fc629cbff38","task_key":"default--test--10998","task_revision_id":"1","upstream_id":"10998","short_description":"Baxter claims that his \"very general way\" of stating what we should strive for…","config":"default","split":"test","body":"{\"category\":\"philosophy\",\"options\":\"['we have unlimited resources to address environmental issues.', 'human satisfaction is the only measure of environmental success.', 'we can measure in some way the incremental units of human satisfaction.', 'we can precisely predict the environmental impact of our actions.', 'the environment is solely a human construct.', 'all living organisms have equal value.', 'all of the above', 'we have a god-like control over the environment.', 'nonhuman animals have no value whatsoever.', 'environmental preservation is the sole responsibility of governments.']\",\"question\":\" Baxter claims that his \\\"very general way\\\" of stating what we should strive for environmentally assumes that\",\"src\":\"ori_mmlu-moral_disputes\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}