{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"df95f179-f7c6-5b4e-9765-603352892be2","task_key":"all--test--df95f179-f7c6-5b4e-9765-603352892be2","task_revision_id":"1","upstream_id":"","short_description":"A plumbing company hired a worker to work at various construction sites. The…","config":"all","split":"test","body":"{\"choices\":\"[\\\"No, because the company's duty to use reasonable care in hiring a competent driver extended only to actions taken by the worker in the scope of his employment.\\\", 'No, because the worker was an independent contractor.', \\\"Yes, because the company fitted the worker's truck with a pipe rack.\\\", \\\"Yes, because the company had a duty to ensure that its workers had valid driver's licenses.\\\"]\",\"question\":\"A plumbing company hired a worker to work at various construction sites. The worker used his own truck to travel between the company's warehouse and the construction sites, but the company fitted the truck with a rack for carrying plumbing pipes. The company paid the worker for traveling between the warehouse and the construction sites, but not for his drive to and from work. Because the worker was required to haul pipes on his truck while driving between the warehouse and the construction sites, the company asked the worker, before hiring him, whether he had a valid driver's license. The worker represented that he did, although in fact his license had been suspended because he had been convicted of recklessly causing motor vehicle collisions. The company made no effort to verify the worker's representation. While driving to work one morning in his truck, the worker carelessly caused a collision in which a woman was injured. In her subsequent action against the plumbing company, based on a theory of negligent hiring, is the woman likely to prevail?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}