{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8d37c64c-adf0-59ec-9ff2-f008e063e7d2","task_key":"default--test--1789","task_revision_id":"1","upstream_id":"1789","short_description":"Client thinks she has been slandered. What of the following is not true about…","config":"default","split":"test","body":"{\"category\":\"law\",\"options\":\"['The accused must have intentionally spread the false statement for it to be considered slander.', 'It is spoken defamation.', 'There are four slander per se categories.', 'Slander is less serious than libel.', 'Plaintiff has to prove special damages, unless it falls into slander per se.', 'The statement does not have to be published if it constitutes slander per se.', 'Slander is only applicable if the plaintiff can prove they suffered emotional distress.']\",\"question\":\"Client thinks she has been slandered. What of the following is not true about slander?\",\"src\":\"ori_mmlu-professional_law\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}