{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"e9447254-76e6-58d6-a7f7-0192894d1997","task_key":"all--test--e9447254-76e6-58d6-a7f7-0192894d1997","task_revision_id":"1","upstream_id":"","short_description":"A customer at a fish market was leaving the store after purchasing an assortment…","config":"all","split":"test","body":"{\"choices\":\"[\\\"admissible, because it is relevant as to the customer's contributory negligence.\\\", \\\"admissible, as a statement of the declarant's present state of mind.\\\", 'inadmissible, as hearsay not within any exception.', 'inadmissible, because it was not a spontaneous utterance.']\",\"question\":\"A customer at a fish market was leaving the store after purchasing an assortment of shrimp, oysters, and scallops. He was walking along the sidewalk in front of the store when he slipped on a piece of eel. He brought suit against the owner of the market claiming that he suffered leg and back injuries. The owner, although admitting that the customer was injured by slipping on the eel, denied negligence and claimed that the customer was contributorily negligent. At trial, the owner calls a witness to testify that before the fall he heard someone call out to the customer, \\\"Watch it, buddy, you're going to step on that piece of fish. \\\"The witness's testimony is\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}