{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu","formal_name":"MMLU","introduction":"MMLU asks four-way multiple-choice questions across 57 subjects, from elementary material to professional-level examinations. Since 2020 it has been the most widely cited single reference point for general capability.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/hendrycks/test","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"da1208fa-5686-5c72-ab4a-84225fda3598","task_key":"all--test--da1208fa-5686-5c72-ab4a-84225fda3598","task_revision_id":"1","upstream_id":"","short_description":"A man allowed his friend to borrow a debit card to buy a computer. When the…","config":"all","split":"test","body":"{\"choices\":\"['No, because the evidence was sufficient to support a verdict of depraved-heart murder.', 'No, because the evidence was sufficient to support a verdict of felony murder.', 'Yes, because the evidence showed that the man did not intentionally pull the trigger.', \\\"Yes, because the evidence showed that the man's intent was not to kill the friend, but to take property from him under a claim of right.\\\"]\",\"question\":\"A man allowed his friend to borrow a debit card to buy a computer. When the friend refused to return the card during a later conversation in a bar, the man pointed a handgun at the friend, held his finger on the trigger, and said, \\\"Give it back!\\\" A bystander screamed and threw a beer bottle at the man. The bottle struck the man's hand and caused the gun to discharge, killing the friend instantly. The man was convicted of murder in a jurisdiction that follows the common law of homicide. On appeal, he argues that the evidence, which essentially set out the facts above, was insufficient to support a verdict of murder. Should the man's conviction be reversed?\",\"subject\":\"professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/hendrycks/test","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}