{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f5c75624-a761-5cf6-b190-7e2d4ee99af2","task_key":"all--test--f5c75624-a761-5cf6-b190-7e2d4ee99af2","task_revision_id":"1","upstream_id":"","short_description":"A shopper purchased a can of sardines, which had been produced by a fish…","config":"all","split":"test","body":"{\"choices\":\"['overrule the objection, because the label is self-authenticating.', 'overrule the objection, because the label is the best evidence showing that the can was produced by the company.', 'sustain the objection, because there is no corroborating evidence that the company placed the label on the can.', \\\"sustain the objection, because the shopper's proper cause of action is for infliction of emotional distress, not strict liability.\\\"]\",\"question\":\"A shopper purchased a can of sardines, which had been produced by a fish company. A short while later at home, the shopper was preparing a sardine sandwich when she opened the can and, to her astonishment, she saw a decomposed mouse. The shopper became nauseated and vomited upon seeing the mouse. After consulting her attorney, the shopper initiated a strict liability action against the company. At trial, the shopper presented into evidence the sardine can with the company's label, along with other corroborating evidence. The attorney representing the company objected to the introduction of the label on the can. The trial court should\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}