{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"edb46328-a975-538e-97ef-3e699c776790","task_key":"all--test--edb46328-a975-538e-97ef-3e699c776790","task_revision_id":"1","upstream_id":"","short_description":"An older and younger brother had always done everything together. They had grown…","config":"all","split":"test","body":"{\"choices\":\"[\\\"violated the younger brother's Sixth Amendment right to counsel.\\\", 'constituted an impermissible conflict of interest.', \\\"violated the younger brother's constitutional rights, but did not infringe on the older brother's constitutional rights.\\\", \\\"did not violate the older brother's constitutional rights if he separately paid for his representation.\\\"]\",\"question\":\"An older and younger brother had always done everything together. They had grown up together, gone to school together, and partied together. One night, while drinking at a bar, both brothers decided that they wanted a little more action than the bar was offering up, so they left and went to a strip club. At the strip club, they paid for private dances and attempted to have conversations with the women. As hard as they tried, the women who worked in the club did not seem as interested in the brothers as they were in the women. As the night wore on, the brothers decided that they would wait behind the club until closing for the women to come out. The victim, who had given a dance to one of the brothers, came out and proceeded to walk to her car. One of the brothers grabbed the victim from behind and pulled her to the ground while the other brother sexually assaulted her. Both brothers were arrested and charged with the rape of the victim. Their long-time family attorney represented them both at their trial. The younger brother intended to testify that his older brother had forced him to hold the victim down by threatening him with a gun. If appropriate objections are made by either brother, the court should rule that the attorney's representation of the brothers at trial\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}