{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1a2083e4-0e58-55ff-ab0d-7137ddb7a7e1","task_key":"all--test--1a2083e4-0e58-55ff-ab0d-7137ddb7a7e1","task_revision_id":"1","upstream_id":"","short_description":"A driver parked his automobile in front of a row of shops, one of which was a…","config":"all","split":"test","body":"{\"choices\":\"['recover, because the store was negligent in failing to inspect its premises and discover dangerous conditions that might harm its invitees.', 'recover, because the driver had intended to purchase merchandise when he entered the store.', 'not recover, because the driver was merely a licensee in the store.', 'not recover, because the store only had an obligation to disclose to invitees any known dangerous conditions on the premises.']\",\"question\":\"A driver parked his automobile in front of a row of shops, one of which was a barber shop where he planned to get a haircut. Realizing that he did not have any change for the parking meter, the driver entered the closest shop, a convenience store, to get change for a $10 bill. After the driver obtained change from one of the clerks in the convenience store, a section of the store's ceiling fell on him as he was about to leave the store. The store had failed to discover the weakened condition of the ceiling. The driver suffered serious head injuries from the falling plaster. In an action by the driver against the convenience store for negligence, the plaintiff will most likely\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}