{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"c6126295-ee22-59d7-b47f-723618a25463","task_key":"default--test--1405","task_revision_id":"1","upstream_id":"1405","short_description":"A woman joined a web site to get her credit reports. The site required creation…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The company will prevail because the provision notifying her of the contract is in bold and the contract is easily accessible.', 'The woman will prevail because terms and conditions have been ruled to be contracts of adhesion that are largely unenforceable.', \\\"The woman will prevail because the terms and conditions, or important parts, must be described on the 'create an account' page, which was not done here.\\\", 'The company will prevail because the woman failed to cancel her subscription and thus implicitly agreed to the terms.', 'The woman will prevail because she did not explicitly agree to the recurring charges.', 'The woman will prevail because she was not given explicit notice of the recurring charges.', 'The company will prevail because the woman did not cancel her subscription and thus accepted the terms.', 'The woman will prevail because the terms were not explicitly stated before she made the initial payment.']\",\"question\":\"A woman joined a web site to get her credit reports. The site required creation of an account before she could enter. On that form, a notice appeared in bold letters: \\\"TERMS AND CONDITIONS: This is a Contract. By creating an account, you agree to and accept the terms and conditions for use of this website.\\\" She did not access the terms, which she could have done by clicking a conspicuous link. She started an account and paid $50 for a credit report. For the next 6 months, her credit card was automatically billed $50, without notice, except that she received email notice that the report was available at the site. The terms and conditions provided that she authorized the charges, unless she notified the company otherwise in advance. Which party will likely prevail on the issue of whether she was bound to pay for the 6 months until she canceled?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}