{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"2255b292-52a5-50c4-85b6-c4bc608b5d24","task_key":"all--test--2255b292-52a5-50c4-85b6-c4bc608b5d24","task_revision_id":"1","upstream_id":"","short_description":"A shopper sued a supermarket for injuries allegedly suffered from a collision…","config":"all","split":"test","body":"{\"choices\":\"[\\\"instruct the jury that it raises the presumption that the brother-in-law's testimony would have been unfavorable to the shopper.\\\", \\\"instruct the jury that it constitutes an adoptive admission that the brother-in-law's testimony would have been unfavorable to the shopper.\\\", \\\"permit the supermarket's attorney to argue that it raises a presumption that the brother-in-law's testimony would have been unfavorable to the shopper.\\\", \\\"neither instruct the jury on the matter nor permit the supermarket's attorney to argue the matter.\\\"]\",\"question\":\"A shopper sued a supermarket for injuries allegedly suffered from a collision with the store's automatic doors. The shopper contended that the doors, which were programmed to swing inward, swung outward and injured him as he attempted to enter the store one afternoon. The shopper's brother-in- law, who was an eyewitness to the accident, was not called to testify at trial. Moreover, the shopper's attorney failed to depose the brother-in-law, who redomiciled out of state shortly after the accident. With respect to the shopper's failure to offer the brother-in-law's testimony at trial, on request by the supermarket's attorney, the court should\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}