{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"46b81d6d-88e7-5dc1-8726-b2c2715e9eb6","task_key":"all--test--46b81d6d-88e7-5dc1-8726-b2c2715e9eb6","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['women gaining the right to vote in many states.', 'an economic downturn that was responsible for the closure of many state institutions.', 'an evolving relationship between the federal government and issues of health and poverty.', 'the rise of voluntary organizations to promote religious and secular reforms.']\",\"question\":\"This question refers to the following information.\\n\\\"I come to present the strong claims of suffering humanity. I come to place before the Legislature of Massachusetts the condition of the miserable, the desolate, the outcast. I come as the advocate of helpless, forgotten, insane and idiotic men and women; of beings, sunk to a condition from which the most unconcerned would start with real horror; of beings wretched in our Prisons, and more wretched in our Alms-Houses. . . .\\n\\\"If my pictures are displeasing, coarse, and severe, my subjects, it must be recollected, offer no tranquil, refined, or composing features. The condition of human beings, reduced to the extremest states of degradation and misery, cannot be exhibited in softened language, or adorn a polished page.\\n\\\"I proceed, Gentlemen, briefly to call your attention to the present state of Insane Persons confined within this Commonwealth, in cages, closets, cellars, stalls, pens! Chained, naked, beaten with rods, and lashed into obedience!\\\"\\n—Dorothea Dix, \\\"Memorial to the Massachusetts Legislature\\\" (1843)\\nDorothea Dix's research and testimony is best understood in the context of\",\"subject\":\"high_school_us_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}