{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e0b6c78f-daf3-55c7-a573-51b150c376d9","task_key":"all--test--e0b6c78f-daf3-55c7-a573-51b150c376d9","task_revision_id":"1","upstream_id":"","short_description":"A young woman joined a social media service that provided networking with other…","config":"all","split":"test","body":"{\"choices\":\"['Yes, because the service breached the duty of good faith inherent in every contract.', 'Yes, because these are common types of damages that are authorized in breach of contract cases.', \\\"No, because Internet laws protect absolutely all communications and downloads of this nature as part of the service's guaranteed scope of free speech rights.\\\", 'No, because humiliation is not an item of damages in a breach of contract case, and the unauthorized collection of personal data does not establish an economic loss.']\",\"question\":\"A young woman joined a social media service that provided networking with other business persons and entities nationwide. The woman discovered that the service was selling her personal profile information, and the information pertaining to thousands of other members, to third party purchasers for tracking of their Internet activities and buying habits. In her class action breach of contract action against the service, she alleged two theories of damages. First, she contended that she and the class members suffered \\\"embarrassment and humiliation\\\" from the disclosure, and second, that she and the others must be compensated for the market value of the information seized. Will the court likely recognize these allegations as sufficient to state a claim for breach of contract?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}