{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4dc2c30c-22eb-5e70-b929-f5c3a40ca31d","task_key":"all--test--4dc2c30c-22eb-5e70-b929-f5c3a40ca31d","task_revision_id":"1","upstream_id":"","short_description":"A wife and her husband were having dinner at a restaurant when the wife excused…","config":"all","split":"test","body":"{\"choices\":\"['recover, because the egg roll on the floor constituted an unsafe condition of the premises.', 'recover, because the egg roll was on the floor for a substantial period of time before the accident.', 'not recover, because the chef did not know that the egg roll was on the floor.', 'not recover, because the customer could have prevented the injury by warning the wife of the presences of the egg roll.']\",\"question\":\"A wife and her husband were having dinner at a restaurant when the wife excused herself to go to the bathroom. The restaurant was owned and operated by a chef. As the wife was walking past a table where another customer was seated, she slipped and fell on an egg roll that had been lying on the floor for quite some time, although the chef was unaware that it had fallen onto the floor. When she fell, her head struck a serving tray that was located in the aisle. The fall caused the wife to suffer a severe concussion. The customer knew that the egg roll was on the floor, and although he could have done so, he did not warn the wife. If the wife asserts a claim against the chef for the injuries she suffered from the fall, she will most likely\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}