{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5e683f5d-fd84-5e33-8fd2-2e24530929d0","task_key":"default--test--5736","task_revision_id":"1","upstream_id":"5736","short_description":"Filibusters are less likely to occur in the House of Representatives than in the…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['the House has fewer members than the Senate', 'House members are less politically diverse than Senate members', 'the House meets for substantially fewer hours per day than does the Senate', \\\"Senate decorum forbids one senator from attempting to stop another's speech\\\", 'the Speaker of the House has the power to end a filibuster at any time', 'debate in the House is in most instances strictly regulated by the rules']\",\"question\":\"Filibusters are less likely to occur in the House of Representatives than in the Senate because\",\"src\":\"ori_mmlu-high_school_government_and_politics\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}