{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"055ee96a-7baf-567b-b9f0-184342595f11","task_key":"default--test--1262","task_revision_id":"1","upstream_id":"1262","short_description":"A manager's competitor sent a defamatory letter to the manager accusing him of…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['No, it will not dismiss because a plaintiff in a defamatory action has an absolute right to a jury trial to prove defamation.', 'Yes, it will dismiss on the grounds that the manager did not suffer any actual harm or damage as a result of the letter.', 'No, it will not dismiss because the manager has the right to sue for emotional damages.', 'No, it will not dismiss because the letter constitutes malicious intent.', \\\"Yes, it will dismiss on the basis that the language is not damaging to the manager's reputation.\\\", 'Yes, it will dismiss because the competitor has freedom of speech.', \\\"No, it will not dismiss because the competitor intentionally sent the letter to harm the manager's reputation.\\\", 'Yes, it will dismiss on the basis that the publication is made to the manager alone.', 'No, it will not dismiss because the circumstances show that all of the elements of defamation are all present.', \\\"Yes, it will dismiss on the grounds that the letter does not specifically mention the manager's profession.\\\"]\",\"question\":\"A manager's competitor sent a defamatory letter to the manager accusing him of professional incompetence and calling him one of the worst businessmen in town. It was addressed to the manager. He read it, put it in a private drawer, and did not read it again. Later, he tried to sue the competitor for defamation as a result of the letter. Will the court likely grant the defendant's motion to dismiss, and on what grounds? Base your answer on the common law definition of defamation.\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}