{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"42f23870-6b71-5018-b1ce-4539d6c1ba0c","task_key":"all--test--42f23870-6b71-5018-b1ce-4539d6c1ba0c","task_revision_id":"1","upstream_id":"","short_description":"A young man called his casual friend to get together after work to go to a…","config":"all","split":"test","body":"{\"choices\":\"['The prosecutor will not file charges because the two men were not related to each other in a way that might create a duty, nor was there a special relationship that would create a duty.', 'The prosecutor will not file charges because the man broke no laws when he walked away and it does not matter what the relationship was between them.', \\\"The prosecutor will file charges because there are eyewitnesses who will testify to the young man's callous behavior to his friend.\\\", 'The prosecutor will file charges because their friendship was a special relationship that created a legal duty for the man to act.']\",\"question\":\"A young man called his casual friend to get together after work to go to a nearby ski lift. They each went up on the lift individually, because neither liked to share. Close to the top of the mountain, the friend fell off of his ski lift, and sustained injuries. The young man then decided he had had enough of skiing, but his friend called out, \\\"...you have to help! You have to call someone! I cant walk and it is your fault I am up here!\\\" The man did not respond and went home. His friend froze to death with no help ever coming to him. The people on the lift behind them both heard and saw everything, and the prosecutors office investigated the incident as a possible negligent homicide. What is the most likely decision of the prosecutor?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}