{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"5e66be96-55a7-5a74-a9bb-8e68e6455f7a","task_key":"all--test--5e66be96-55a7-5a74-a9bb-8e68e6455f7a","task_revision_id":"1","upstream_id":"","short_description":"A minor child was eating a fish sandwich in a fast food restaurant when a large…","config":"all","split":"test","body":"{\"choices\":\"['No, because even without negligence there is strict liability imposed for selling an unreasonably dangerous product.', 'Yes, because a customer voluntarily assumes the foreseeable risk that fish will contain metal fish hooks.', 'Yes, because customers are expected to know that a fish sandwich will often have foreign objects in it.', 'No, because the seller did not post a warning that its fish might contain fish hooks.']\",\"question\":\"A minor child was eating a fish sandwich in a fast food restaurant when a large metal object, later identified as a fish hook, got caught in her throat. The trapped hook caused her to cough blood uncontrollably, and she was rushed to a hospital for emergency surgery. The restaurant later defended the customer's lawsuit by showing that it had done everything reasonable to eliminate foreign objects from its product and there was no negligence. Will the lawsuit be dismissed and judgment entered for the restaurant?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}