{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"70c700bd-3d47-5fb3-9082-a0628df7e15d","task_key":"default--test--1747","task_revision_id":"1","upstream_id":"1747","short_description":"A businessman living was an avid baseball fan who frequently traveled to a…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['discharge by impossibility.', 'constructive condition precedent that was substantially performed.', 'modification.', 'material breach of contract.', 'minor breach of contract.', 'constructive condition subsequent.', 'discharge by impracticability.', 'breach of contract.', 'discharge by frustration of purpose.', 'anticipatory repudiation.']\",\"question\":\"A businessman living was an avid baseball fan who frequently traveled to a casino and placed wagers on baseball games. One October, his beloved baseball team was playing in the playoffs and he wanted to place a large bet on his team. He told one of his employees that he wanted to bet $10,000 on his team to win the championship, but that he did not have the time to travel to the casino to place the bet. He then told his employee that he wanted the employee to travel to the casino and place the bet for him. The businessman and the employee agreed that the employee would travel to the casino, place a $10,000 bet for the businessman and then immediately return. In exchange, the employee would be paid $500. The parties reduced this agreement to writing. After the agreement was signed, the businessman handed the employee $10,000 and a round-trip airline ticket. The businessman then told the employee that he would be paid $500 after the employee returned. The employee arrived the next day and immediately went to the casino. There he noticed that the marquis in the parking lot was unlit and covered in a banner that read, \\\"Casino Temporarily Closed Due to Renovations. \\\" Unable to reach the businessman by telephone, the employee placed the bet at another gaming establishment located next door to the casino. The following morning, the employee flew back and gave the betting receipt to the businessman. When the businessman saw that the bet had been made at another gaming establishment, he angrily told the employee, \\\"I purposely directed you to wager that $10,000 at the casino. Since you failed to follow my instructions, the deal's off. \\\" As a result, the businessman refused to pay the employee the $500. If the employee initiates suit for breach of contract, the court will most likely determine that placing the $10,000 wager at the other gaming establishment, rather than at the casino, constituted a\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}