{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"dd254714-4b9e-539c-8462-48834ef49aa5","task_key":"all--test--dd254714-4b9e-539c-8462-48834ef49aa5","task_revision_id":"1","upstream_id":"","short_description":"Two women lived in a one-bedroom house with 20 dogs. A dog-walker who entered…","config":"all","split":"test","body":"{\"choices\":\"['Yes, because there was nothing that was done by the humane society that involved police action.', 'Yes, because the SPCA personnel did not at any time act \\\"under color of law\\\" in their actions.', 'No, because the SPCA was organized under a non-profit certificate of incorporation that was issued by the state.', 'No, because the SPCA acted under the authority of the state statute and cooperated with state authorities to perform the authorized functions.']\",\"question\":\"Two women lived in a one-bedroom house with 20 dogs. A dog-walker who entered the house was appalled by their deplorable living conditions. The animals' excrement was deposited throughout the house, they were kept in tiny pens, and some of the animals were sick. The visitor called the SPCA, a private non-profit humane society. State law authorized humane societies to receive abused or neglected animals from animal control officers, to care for them and adopt them out. The SPCA took the animals designated by the officers. The officers then arrested the women for animal neglect. At a hearing, the prosecutor was unprepared, and the judge dismissed the charges. The women then sued the officers and the SPCA under 42 U.S.C. § 1983 for civil rights violations. The SPCA defendants filed a motion to dismiss based on the absence of the \\\"state action\\\" on their part. Will the court likely grant the motion to dismiss?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}