{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1863b37a-e5a3-5903-b5e4-28cb87cb8761","task_key":"all--test--1863b37a-e5a3-5903-b5e4-28cb87cb8761","task_revision_id":"1","upstream_id":"","short_description":"This question refers to the following information.","config":"all","split":"test","body":"{\"choices\":\"['was necessary for the German people to become reconciled to their own history', 'hindered the healing process for the German people', 'would ensure that it never occurred again', 'would allow the Jews to forgive the German people']\",\"question\":\"This question refers to the following information.\\nAs a Jew, I have never believed in collective guilt. Only the guilty were guilty.\\nChildren of killers are not killers but children. I have neither the desire nor the authority to judge today's generation for the unspeakable crimes committed by the generation of Hitler.\\nBut we may—and we must—hold it responsible, not for the past, but for the way it remembers the past. And for what it does with the memory of the past. In remembering, you will help your own people vanquish the ghosts that hover over its history. Remember: a community that does not come to terms with the dead will continue to traumatize the living.\\nWe remember Auschwitz and all that it symbolizes because we believe that, in spite of the past and its horrors, the world is worthy of salvation; and salvation, like redemption, can be found only in memory.\\nElie Wiesel, \\\"Reflections of a Survivor,\\\" 1987\\nFrom the passage, one may infer that Wiesel asserted that remembering the Holocaust\",\"subject\":\"high_school_european_history\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}