{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"bf053436-cebf-5bf4-b651-af7b469c9b25","task_key":"default--test--1157","task_revision_id":"1","upstream_id":"1157","short_description":"Proposed legislation was offered to a state legislature that would reorganize…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The equal protection clause of the Fourteenth Amendment.', 'The speech and debate clause.', 'The right to a jury trial as guaranteed by the Sixth Amendment.', 'The due process clause of the Fourteenth Amendment.', 'The establishment clause of the First Amendment.', 'The protection from ex post facto laws.', 'The privileges and immunities clause of the Fourteenth Amendment.', 'The right of assembly as guaranteed by the First Amendment.', 'The cruel and unusual punishment clause of the Eighth Amendment.', 'The right to petition as guaranteed by the First Amendment.']\",\"question\":\"Proposed legislation was offered to a state legislature that would reorganize the state police. The bill created a great deal of controversy, both in and outside the state government. Several leaders of the minority party in the legislature decided to oppose the legislation. One member of the minority party disagreed with his party's opposition to the bill and publicly announced his support for the legislation. The minority party leaders called a caucus to discuss and determine their legislative strategy for floor debate on the bill. When the disagreeing member appeared at the door of the caucus room, he was denied admission because of his anti-party stance. He was also informed that he would be removed from all of his committee assignments. During the caucus, the party members discussed other means of disciplining the member for his party insubordination. It was suggested that they issue a press release in which the party would publicly castigate him for his actions. The leader of the party said that \\\"the member is a cutthroat politician who is only looking out for where his next buck will come from. \\\"Which of the following constitutional provisions would give the ousted member his best grounds for challenging his exclusion from the party caucus? \",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}