{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"f2ad8677-3978-5326-a8f3-61d43c2c348b","task_key":"all--test--f2ad8677-3978-5326-a8f3-61d43c2c348b","task_revision_id":"1","upstream_id":"","short_description":"A fumigation company was hired to eliminate pests in one of two buildings in a…","config":"all","split":"test","body":"{\"choices\":\"['No, because the condominium complex owners were responsible for accurately conveying the condition of their buildings.', 'No, because the fumigation company exercised a high level of care.', 'Yes, because the fumigation company can be held strictly liable for its activity.', 'Yes, because the fumigation company put a dangerous product into the stream of commerce.']\",\"question\":\"A fumigation company was hired to eliminate pests in one of two buildings in a condominium complex that shared a common wall. The owners of the complex told the fumigation company that the common wall separating the infested building from the uninfested building was an impenetrable fire wall. The fumigation company did its own thorough inspection and determined that the buildings were indeed completely separated by the wall. Residents of the condominium units in the building that was to be sprayed were told to evacuate, but the residents of the uninfested building were told that they could remain while the other building was treated. During and shortly after the fumigation, in which a highly toxic chemical was used, many residents of the uninfested building became sick. It was determined that their illnesses were caused by the fumigation chemical. In fact, there was a hole in the fire wall separating the two buildings, but because it could only be observed from a specific position in the crawl space underneath the floor of the uninfested building, it had not been discovered by either the fumigation company or any previous building inspector. Are the residents of the uninfested building likely to prevail in a tort action against the fumigation company?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}