{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"82d08f48-b358-580a-a6e4-98966b15fa1e","task_key":"default--test--1445","task_revision_id":"1","upstream_id":"1445","short_description":"On the basis of scientific studies showing a causal relationship between the…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"[\\\"Yes, because it is directly related to the government's interest in preventing health risks.\\\", 'No, because it unfairly targets a specific industry.', 'Yes, because it only restricts commercial advertising, not personal speech.', 'No, because it is more extensive than necessary to serve the government interest in preventing certain cancers.', 'Yes, because it restricts potentially harmful information from reaching the public.', 'No, because it does not provide an alternative method for meat producers to advertise their products.', 'No, because it does not serve a substantial government interest.', 'Yes, because it serves a legitimate government interest in protecting public health.', 'Yes, because it does not affect speech protected by the First Amendment.', \\\"No, because it violates the producers' right to commercial speech.\\\"]\",\"question\":\"On the basis of scientific studies showing a causal relationship between the consumption of \\\"red meat\\\" (principally beef) and certain forms of cancer, a federal statute prohibits all commercial advertising of red meat products. The statute does not, however, restrict the sale of red meat products. Producers of red meat have challenged the statute as a violation of their free speech rights protected by the First Amendment. Is the court likely to find the statute constitutional?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}