{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0430245d-ba84-5269-9384-c025099bf392","task_key":"default--test--1610","task_revision_id":"1","upstream_id":"1610","short_description":"A writer sent a four-page synopsis of an idea for a new television series to a…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"The court will rule that there can be no contract as the writer didn't explicitly state his expectation for compensation for the idea itself.\\\", 'The court will rule that there was an implied in fact contract between the parties based on the conduct that they manifested.', 'The court will rule that there was no consideration specifically mentioned and under those circumstances, the law viewed the synopsis as a gift to the producer.', 'The court will rule that there is an express contract because there was a meeting of the mind and mutual assent to the basic terms.', 'The court will rule that there was a unilateral contract, with the producer obligated to pay only if he used the idea.', \\\"The court will rule that the writer's case is invalid as there was no written agreement.\\\", \\\"The court will rule that the producer's assistant lacked the authority to form a contract, and thus, no contract exists.\\\", 'There were too many terms left out for this to be a contract, and it was too indefinite for the court to imply what the terms might have been.', \\\"The court will rule that the phone call between the writer and the producer's assistant formed a verbal contract.\\\"]\",\"question\":\"A writer sent a four-page synopsis of an idea for a new television series to a Hollywood producer. He sent it in response to an ad for new ideas in an industry publication. He discussed it with the producer's assistant in a phone call, and mentioned his expectation of compensation. She said, \\\"Well, of course, we always pay for a writer's work.\\\" She said she would go over it with her boss to see if he liked it. Several months later, the writer saw a casting call for a new series. The plot and characters were nearly identical to those described in his synopsis. He sued the producer for breach of contract. The producer defended by arguing that there was no contract. What is the likely ruling of the court?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}