{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4b87b2b1-9c96-50fa-b3c5-9b49d3b1bb96","task_key":"default--test--5063","task_revision_id":"1","upstream_id":"5063","short_description":"You seek the counsel of one of your peers who designed and implemented a…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['Kill them with kindness', 'Ignore their concerns', 'Expose them to ridicule', 'Bribe them into agreement', 'Use aggressive tactics', 'Solve their problems', 'Manipulate their emotions', 'Overwhelm them with facts', 'Overpower them with authority', 'Intimidate them with legal jargon']\",\"question\":\"You seek the counsel of one of your peers who designed and implemented a communication program that resulted in the successful permitting of an automotive recycling center, despite strong opposition from community organizations. You have to design a program to win approval for a similar project proposed by your company. Your peer gives you a brief piece of advice that succinctly describes the foundation of his successful program. What is most likely your peer's advice for a successful communication program?\",\"src\":\"ori_mmlu-public_relations\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}