{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"381bbf43-945d-58f1-88be-1a52e002aac0","task_key":"all--test--381bbf43-945d-58f1-88be-1a52e002aac0","task_revision_id":"1","upstream_id":"","short_description":"A shopper in a grocery store was injured when she slipped and fell on a puddle…","config":"all","split":"test","body":"{\"choices\":\"[\\\"The objection will be overruled because the asst. manager's statement was not hearsay \\\\x80\\\\x94 it was offered to show that they were aware of a problem on the floor.\\\", \\\"The objection will be sustained because the asst. manager's statement was pure hearsay and she had to be available for cross-examination for the statement to be admissible.\\\", \\\"The objection will be sustained because the statement was irrelevant to whether or not the store was negligent and liable for the shopper's injuries.\\\", \\\"The objection will be overruled because the asst. manager's statement, made in the scope of her duties, was a vicarious admission of the party opponent, the grocery store.\\\"]\",\"question\":\"A shopper in a grocery store was injured when she slipped and fell on a puddle of liquid on the floor, causing her serious injuries. She sued the store for damages. She testified at trial that while sitting on the floor trying to gather the strength to get up, she heard a grocery employee asst. manager say to an employee, \\\"I thought I told you to get that mess up!\\\" The grocer's counsel objected on the basis of hearsay to the admissibility of the purported statements of the asst. manager and the other employee. What will the court decide?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}