{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"01f72e8d-4dbe-5982-a288-4c122c96c322","task_key":"default--test--1765","task_revision_id":"1","upstream_id":"1765","short_description":"An art collector attended a party on March 15. At the party, the art collector…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['An enforceable contract does not exist between the parties, because of mutual mistake of fact.', 'There is no enforceable contract, because the woman did not accept the initial offer in writing.', 'The art collector is obligated to sell the woman the painting for $10,000, because the woman, as offeree, had the right to accept the initial offer within the 30-day period.', 'The art collector is obligated to sell the painting for $10,000 as the woman accepted the original offer before it was revoked.', 'The woman is obligated to pay the art collector $10,000, because the art collector did not properly revoke the initial offer.', 'Since the parties entered into a subsequent modification, the woman is obligated to pay the art collector $15,000 for the painting.', 'The woman is not obligated to pay any amount, because the initial offer was not legally binding.', 'The woman is obligated to pay $15,000, because she agreed to this price during their telephone conversation on April 10.', 'The woman is obligated to pay $15,000 as the new agreement made on April 10 supersedes the initial offer.', 'Since the art collector, as offeror, had the power to revoke the original offer before acceptance, the woman is obligated under the terms of their April 10 agreement.']\",\"question\":\"An art collector attended a party on March 15. At the party, the art collector was describing his collection to a woman in attendance. When the art collector described a painting by a well-known artist, the woman indicated she might like to buy that painting. The art collector said, \\\"I'll sell you the painting for $10,000. I'll give you 30 days to decide whether you want the painting. \\\" On March 25, the art collector wrote to the woman and stated that the offer of March 15 was withdrawn. The woman received the March 25 letter on March 26. On March 27, the woman wrote the art collector the following letter:\\\"Please be advised that I hereby accept your offer of March 15. \\\" The art collector received this letter on March28. Thereafter, the art collector and the woman engaged in several telephone discussions. On April 10, the woman, in a telephone conversation, told the art collector that she would pay $15,000 if the painting was delivered on or before April 15. The art collector agreed to deliver the painting for $15,000. On April 15, the art collector tendered the painting, but the woman refused to pay more than $10,000. If the art collector asserts a claim against the woman for breach of contract, which of the following is the most accurate statement?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}