{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6a237ae6-8941-5ee3-b859-fef56bb78bbf","task_key":"default--test--5067","task_revision_id":"1","upstream_id":"5067","short_description":"House members are more concerned about their committee assignments than are…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['House members may serve on only one committee at a time', 'bills in the Senate must pass through all committees before reaching the floor', 'Senate committees have relatively little power to alter legislation', 'Senators are able to serve on multiple committees at once', 'House members have less time to devote to committee work due to their shorter terms', 'the rules make it extremely difficult for representatives to influence legislation on the House floor', 'Senate committees do not have as much influence on the final legislative outcome as House committees', 'Senators have more power in the legislative process than House members']\",\"question\":\"House members are more concerned about their committee assignments than are senators because\",\"src\":\"ori_mmlu-high_school_government_and_politics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}