{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"800f6170-330e-58b8-90c0-79ebb181722f","task_key":"default--test--1865","task_revision_id":"1","upstream_id":"1865","short_description":"A father lived with his son, who was addicted to crack cocaine. Under its…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['Give the self-defense instruction, because the son forcibly entered the home.', 'Deny the self-defense instruction, because the son was not actually under the influence of drugs.', \\\"Give the self-defense instruction, because it expresses the defense's theory of the case.\\\", 'Deny the self-defense instruction, because the father had other options to protect himself.', 'Give the self-defense instruction, because the evidence was sufficient to raise the defense.', 'Deny the self-defense instruction, because the son was unarmed.', 'Deny the self-defense instruction, because the father used excessive force.', 'Deny the self-defense instruction, because the father was not in imminent danger from his son.']\",\"question\":\"A father lived with his son, who was addicted to crack cocaine. Under its influence, the son often becameviolent and physically abused his father. As a result, the father always lived in fear. One night, the fatherheard his son on the front stoop making loud obscene remarks. The father was certain that his son wasunder the influence of crack and was terrified that he would be physically beaten again. In his fear, hebolted the front door and took out a revolver. When the son discovered that the door was bolted, he kickedit down. As the son burst through the front door, his father shot him four times in the chest, killing him. Infact, the son was not under the influence of crack or any drug and did not intend to harm his father. At trial, the father presented the above facts and asked the judge to instruct the jury on self-defense. How should the judge instruct the jury with respect to self-defense?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}