{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"63bb3153-e4bc-55af-b941-912889eccaca","task_key":"all--test--63bb3153-e4bc-55af-b941-912889eccaca","task_revision_id":"1","upstream_id":"","short_description":"An elevator company installed an elevator in an apartment building. Under the…","config":"all","split":"test","body":"{\"choices\":\"['recover, because she was an invitee of a tenant in the building.', \\\"recover, because the owner would be vicariously liable for the company's failure to inspect the elevator.\\\", 'not recover, because the woman was negligent in failing to exit the elevator on the 12th floor when the other occupants exited the elevator.', 'not recover, because there had been no previous assaults in the apartment building.']\",\"question\":\"An elevator company installed an elevator in an apartment building. Under the terms of its contract with the owner of the apartment building, the company agreed to carry out regular monthly inspections of the elevator and to make all necessary repairs. The elevator was installed in the apartment complex in January. For the next 10 months, the company made regular inspections. However, starting in November, the company began to have some difficulties with its employees and failed to send inspectors out for three successive months. On February 20, a woman went to visit her friend, who was a tenant on the 20th floor of the building. The woman used the intercom to call up to the friend, who buzzed to unlock the front door so that the woman could enter the building. The woman headed to the elevator. When the doors opened, several people were already inside the elevator, apparently having entered the elevator from the parking garage below the building. One of the people in the elevator was a man who acted shifty and nervous. By the 12th floor, only four persons were left, including the woman and the man. At that floor, the other two people left, leaving the woman and the man alone on the elevator. Apprehensive at finding herself alone with such an unkempt stranger, the woman pressed the 13th floor button, intending to step out and hoping to find other persons on that floor. Unfortunately, the elevator stalled, and came to a stop between the 12th and 13th floors. The woman's fears were justified, as the man suddenly grabbed at her purse. When the woman resisted, the man struck her, and she fell unconscious. When the woman recovered consciousness, she found herself in a hospital with a concussion and multiple contusions on her body. She later learned that the elevator had remained stalled for 50 minutes. Thereafter, it moved to the 20th floor, where she was found by her friend, who took her to the hospital. Two weeks later, the police arrested the man and found the woman's purse in his possession. This was the first assault that had occurred in the apartment building. The elevator's stalling was caused by a defective cable that a routine inspection would have discovered. If the woman brings suit against the owner of the apartment building for negligence, she will most likely\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}