{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ada51ba6-c069-5f0d-bded-4ffe2d71face","task_key":"all--test--ada51ba6-c069-5f0d-bded-4ffe2d71face","task_revision_id":"1","upstream_id":"","short_description":"A certified airline pilot owned a single-engine Cessna plane. One afternoon, the…","config":"all","split":"test","body":"{\"choices\":\"[\\\"recover, because a reasonable person would have been distressed by the pilot's actions.\\\", \\\"recover, because the pilot's conduct was extreme and outrageous.\\\", 'not recover, because the pilot intended his actions as a practical joke and, therefore, lacked sufficient intent for an intentional tort.', 'not recover, because the girlfriend did not suffer any physical injury.']\",\"question\":\"A certified airline pilot owned a single-engine Cessna plane. One afternoon, the pilot invited his girlfriend, to go flying with him. The girlfriend, who was scared of flying, reluctantly agreed. During the flight, the pilot decided to play a practical joke and shut off the plane's engine. When the plane went into a nosedive, the pilot said, \\\"Oh my God, we're going to crash. \\\" The girlfriend became panic- stricken and started screaming hysterically. A few seconds later, the pilot re-started the engine and said, \\\"Don't worry €”I was only joking. \\\" The plane landed safely and neither person was injured. If the girlfriend brings suit against the pilot for intentional infliction of emotional distress, she will probably\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}