{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5aa4e35a-5550-5b0f-8120-8c39bf4b8f2d","task_key":"all--test--5aa4e35a-5550-5b0f-8120-8c39bf4b8f2d","task_revision_id":"1","upstream_id":"","short_description":"An avid stamp collector had amassed a large and valuable collection. Many of his…","config":"all","split":"test","body":"{\"choices\":\"['The price term is $75,000, and it is enforceable.', 'The price term is $78,000, and it is enforceable.', 'The price term is $75,000, but the court will reform the price to $78,000 in order to correct a mistake in integration.', 'The price term is $75,000, but either party can rescind the contract, because there was a mutual mistake as to a basic assumption of fact.']\",\"question\":\"An avid stamp collector had amassed a large and valuable collection. Many of his stamps were extremely rare and highly coveted. On numerous occasions the collector had rejected offers to sell his prized collection. Finally, on December 1, the collector sent his cousin, who was also a stamp collector, the following letter:\\\"I've decided to part with my stamp collection. Since I want to keep it in the family, I'll sell you my entire collection for $75,000. \\\"The collector had dictated this letter to his secretary, who mistakenly typed \\\"$75,000,\\\" instead of $78,000 that the collector had specified. After typing the letter, the secretary gave it to the collector, who hastily signed it without noticing the mistaken price. The cousin received this letter on December 3. On December 5, the cousin sent a letter accepting the collector's offer. Which of the following correctly states the agreement's price term and its legal effect?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}