{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"70623490-3e33-5bc9-8de2-688674f0c743","task_key":"all--test--70623490-3e33-5bc9-8de2-688674f0c743","task_revision_id":"1","upstream_id":"","short_description":"A dance student worked very hard to strengthen her legs, including performing…","config":"all","split":"test","body":"{\"choices\":\"['Yes, because it is as an exception to hearsay as a spontaneous declaration to an opponent-party.', 'Yes, because the building owner can testify and be questioned about what the janitor told him.', 'No, because it is irrelevant and immaterial.', 'No, because it is hearsay within hearsay, and there are no hearsay exceptions that apply.']\",\"question\":\"A dance student worked very hard to strengthen her legs, including performing additional activities such as leaping and bounding up multiple flights of stairs in her apartment building. While she was performing her exercises, one of the janitors in the building warned her that \\\"she was bound to get hurt if she keeps it up, because there are problems with the floor.\\\" After falling and breaking a leg due to a loose floorboard, she brought suit against the building owner. The building owner wants to testify that: \\\"The janitor told me 'I told her that running too fast is dangerous and she was bound to get hurt.' \\\". Is the building owner's statement likely to be held admissible?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}