{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b1c160c8-50f9-5643-8da9-734b7fa2c6a5","task_key":"all--test--b1c160c8-50f9-5643-8da9-734b7fa2c6a5","task_revision_id":"1","upstream_id":"","short_description":"A man worked as a clerk in a gourmet coffee shop. His friend often stopped by…","config":"all","split":"test","body":"{\"choices\":\"[\\\"prevail, because the company would be vicariously liable for the man's unauthorized actions.\\\", \\\"prevail, because the friend's negligence would be imputed to the company.\\\", 'not prevail, because the man acted outside the scope of employment by entrusting the supervision of the shop to his friend.', \\\"not prevail, because the company is not liable for the friend's negligence.\\\"]\",\"question\":\"A man worked as a clerk in a gourmet coffee shop. His friend often stopped by the shop to chat with the man while the man was working. One afternoon, while the friend was visiting the shop, the man needed to use the bathroom. There were no customers in the shop at that moment, so the man asked the friend to watch the store for him while he went in the back to use the bathroom. The friend agreed to do so. While the man was in the bathroom, a customer came into the store. She wanted to purchase some freshly ground coffee. The shop had a grinding machine that customers could use. The customer selected a bag of coffee beans and then took it to the grinding machine. She was unsure how to start the machine, so she asked the friend for help. The friend tried to show the customer how to use the machine, but he was not familiar with how it worked, and he gave the customer erroneous instructions. The customer tried to follow the friend's instructions, but this led to her getting her finger pinched and cut by the machine. If the customer asserts a claim against the coffee shop company for her finger injury, the customer will most likely\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}