{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"edeaa489-6b68-52d6-acaf-ba27a8b12bce","task_key":"all--test--edeaa489-6b68-52d6-acaf-ba27a8b12bce","task_revision_id":"1","upstream_id":"","short_description":"A man and a woman had a romantic relationship for one year, but the man ended…","config":"all","split":"test","body":"{\"choices\":\"['prevail, because the telephone calls intruded upon his seclusion and solitude.', \\\"prevail, but only if the man is able to prove malice on the defendant's part.\\\", 'not prevail, because the telephone calls did not cause the man to suffer any economic loss or hardship.', \\\"not prevail, unless the woman's conduct was a crime in some respect.\\\"]\",\"question\":\"A man and a woman had a romantic relationship for one year, but the man ended it. The woman was unable to accept the breakup and continued trying to contact the man. The woman would call the man's home and cellphone many times each day, and also tried to visit the man's home and workplace frequently. The man tried to discourage the woman's behavior, but the woman persisted. If the man asserts a claim against the woman based on invasion of privacy, the most likely outcome is that the man will\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}