{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"13814906-d593-5c5c-b38d-6e342e2417ad","task_key":"default--test--1545","task_revision_id":"1","upstream_id":"1545","short_description":"A laboratory manufactures nitroglycerin (a heavy, oily, explosive, poisonous…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The laboratory had previously survived similar incidents without any damage to surrounding properties.', 'The laboratory used extraordinary care in the manufacture and storage of nitroglycerin and was not guilty of any negligence that was causally connected with the explosion.', 'The artist did not have appropriate safety measures in place to protect their artwork.', 'The laboratory had all necessary permits and licenses to manufacture nitroglycerin.', \\\"The harm that the artist suffered would not have resulted but for the abnormal fragility of the artist's work.\\\", 'The artist had previously accepted compensation for potential risks due to their proximity to the laboratory.', 'The artist had been warned about potential risks due to their proximity to the laboratory.', \\\"The laboratory was not aware of the artist's studio being in close proximity to their plant.\\\", 'The laboratory has a contract with the federal government whereby all the nitroglycerin manufactured at its plant is used in U. S. military weapons systems.']\",\"question\":\"A laboratory manufactures nitroglycerin (a heavy, oily, explosive, poisonous liquid used chiefly in making dynamite) at its main plant. An artist who specialized in making delicate glass sculptures had a studio two blocks away from the laboratory's plant. Late one evening, there was an explosion at the laboratory's plant. The force of the explosion caused the artist's studio to be shaken, which resulted in the destruction of valuable artwork in the studio. The artist now asserts a tort action against the laboratory to recover damages. Which of the following, if established, would furnish the laboratory with its best possible defense?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}