{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"69c60292-3c75-5f3a-8078-39c707ad7a03","task_key":"default--test--1726","task_revision_id":"1","upstream_id":"1726","short_description":"A business purchased a computer from a seller's web site. The purchase form had…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['No, because federal law requires that a company give 90 days for return and the 30-day provision was null and void.', \\\"No, because there was no contract formed due to the purchaser's lack of assent to the additional terms and conditions.\\\", 'Yes, because by not returning the computer within 30 days, the purchaser was bound to the additional terms and conditions that were a part of the contract.', 'Yes, because the seller provided the additional terms and conditions in writing with the delivered product.', 'No, because the purchaser did not explicitly agree to the terms and conditions that include private arbitration.', 'Yes, because the purchaser was aware of the additional terms and conditions, despite not accessing them.', 'No, because the seller did not clearly communicate the additional terms and conditions to the purchaser.', 'Yes, because the additional terms and conditions were part of the contract, regardless of whether the purchaser read them or not.', 'No, because the seller did not obtain explicit consent from the purchaser for the additional terms and conditions.', 'Yes, because federal law requires that arbitration must be used to attempt an initial resolution with respect to sales that take place on the internet.']\",\"question\":\"A business purchased a computer from a seller's web site. The purchase form had a prominent hyperlink connection to a page that contained \\\"important additional terms and conditions of sale,\\\" but the purchaser did not access that page. The delivered product included the additional terms in writing, along with a bold-lettered notice requiring the purchaser to return the product within 30 days if it did not accept the stated terms and conditions. After 35-days, the business became dissatisfied with the computer, and sued the seller for breach of contract. The seller demanded that the dispute be decided by private arbitration as per the additional terms and conditions to the agreement. Is the purchaser legally bound to participate in arbitration?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}