{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fbec8052-39c1-5559-8a30-a65291adb93b","task_key":"default--test--1244","task_revision_id":"1","upstream_id":"1244","short_description":"A football team entered into a 10-year lease with a city for use of the city's…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"The court should first hold a nonjury trial of the city's claim without giving binding effect to its findings or conclusions in the later jury trial of the team's counterclaim.\\\", \\\"The court should first hold a jury trial of the team's counterclaim, and its findings should be binding in the later nonjury trial of the city's claim.\\\", \\\"The court should first hold a nonjury trial of the team's counterclaim, and then a jury trial of the city's claim.\\\", \\\"The court should schedule a jury trial of both the city's claim and the team's counterclaim.\\\", \\\"The court should hold a simultaneous jury trial of the team's counterclaim and nonjury trial of the city's claim.\\\", \\\"The court should first hold a jury trial of the city's claim, and then a nonjury trial of the team's counterclaim.\\\", \\\"The court should first hold a jury trial of the team's counterclaim, and then a nonjury trial of the issues remaining in the city's claim.\\\", \\\"The court should schedule a nonjury trial of both the city's claim and the team's counterclaim.\\\", \\\"The court should first hold a nonjury trial of the city's claim, and then a jury trial of the issues remaining in the team's counterclaim.\\\", \\\"The court should first hold a nonjury trial of the city's claim, and its findings should be binding in the later jury trial of the team's counterclaim.\\\"]\",\"question\":\"A football team entered into a 10-year lease with a city for use of the city's athletic stadium. Five years into the lease, the team threatened to leave the stadium and move to another city. The city sued the team in federal court, seeking a permanent injunction to prevent the team from breaching its lease and leaving. In its answer, the team included a counterclaim seeking $10 million in damages for losses caused by the city's alleged failure to properly maintain the stadium, as the lease required. The team demanded a jury trial on the counterclaim. The city moved to try its claim for a permanent injunction before the trial on the team's counterclaim. The team objected and moved that the jury trial of its counterclaim be held before the trial of the city's injunction claim. How should the court rule on the parties' motions?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}