{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"83b74fd8-3b5e-5f99-9943-a65aee5a5531","task_key":"default--test--2153","task_revision_id":"1","upstream_id":"2153","short_description":"The best conclusion that can be drawn about the four-day ( compressed) workweek…","config":"default","split":"test","body":"{\"category\":\"psychology\",\"options\":\"['negative effects on productivity but little or no impact on attitudes.', 'positive effects on productivity but negative effects on attitudes.', 'long-lasting negative effects on attitudes and productivity.', 'positive effects on attitudes but little or no impact on productivity.', 'positive effects on productivity but little or no impact on attitudes.', 'negative effects on attitudes but positive effects on productivity.', 'no significant effects on either attitudes or productivity.', 'negative effects on attitudes initially but, over time, increasingly positive effects on both attitudes and productivity.', 'positive effects on attitudes initially but, over time, increasingly negative effects on both attitudes and productivity.', 'long-lasting positive effects on attitudes and productivity.']\",\"question\":\"The best conclusion that can be drawn about the four-day ( compressed) workweek is that it has:\",\"src\":\"ori_mmlu-professional_psychology\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}