{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0eda28d6-c3b1-5c4d-9f31-ccffe0e9c8f8","task_key":"all--test--0eda28d6-c3b1-5c4d-9f31-ccffe0e9c8f8","task_revision_id":"1","upstream_id":"","short_description":"After the murder of his brother, a man published the following notice in the…","config":"all","split":"test","body":"{\"choices\":\"['succeed, because his apprehension of the murderer created a contract implied in law.', 'succeed, because he was unaware of the offer.', 'not succeed, because he did not have knowledge of the reward.', 'not succeed, because his investigation was not a bargained-for exchange.']\",\"question\":\"After the murder of his brother, a man published the following notice in the local newspaper:\\\"REWARDAny person who supplies information leading to the arrest and conviction of the murderer of my brother will be paid $10,000. \\\"An amateur detective, without knowledge of the reward notice, began investigating the matter as a result of his own curiosity kindled by the sensationalism surrounding the brother's murder. One week later, the detective secured information that led to the arrest and later conviction of the murderer. During the murder trial, the detective found out about the reward and demanded the $10,000 from the man. In an action by the detective to recover the $10,000 reward, he will\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}