{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d343fdb0-b647-54d8-97a2-1bd43fb6dd06","task_key":"all--test--d343fdb0-b647-54d8-97a2-1bd43fb6dd06","task_revision_id":"1","upstream_id":"","short_description":"A chemical company's plant was located in a residential community. The…","config":"all","split":"test","body":"{\"choices\":\"['No, because the chemical company conformed to the requirements of reasonable care and to the applicable government regulations.', 'No, because the chemical company used a state-of-the-art storage tank.', 'Yes, because the chemical company is strictly liable in tort for any harm caused by the toxic chemicals it produced.', 'Yes, because the storage of toxic chemicals in a residential community created a highly significant risk of physical harm even when reasonable care was exercised.']\",\"question\":\"A chemical company's plant was located in a residential community. The manufacturing process used at the plant generated a toxic chemical as a by- product. The chemical was stored in a state-of-the- art tank on the site before being moved to an off-site disposal facility. The on-site storage arrangement conformed to the requirements of reasonable care and to the applicable government regulations. However, the storage of the toxic chemical created a foreseeable and highly significant risk of physical harm even when reasonable care was exercised. Despite the chemical company's proper use and care of the storage tank, toxic fumes escaped from the tank and made residents of the area seriously ill. No state or federal statutes address the issue of the company's liability. In an action by one of the affected residents against the chemical company, will the resident be likely to prevail?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}