{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"9ffd279d-9de0-51e3-92da-e3b880de95f2","task_key":"default--test--5059","task_revision_id":"1","upstream_id":"5059","short_description":"What was the significance of the Gulf of Tonkin resolution?","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['It triggered a global protest against US involvement in Vietnam', 'It resulted in a significant reduction of US troops in Vietnam', 'It allowed the US to intensify its involvement in Vietnam', 'It illustrated the influence of public opinion on US foreign policy', 'It marked the beginning of diplomatic negotiations with Vietnam', 'It enhanced Congressional control over the Vietnam War', 'It curtailed US involvement in Vietnam', 'It established the neutrality of the United States in the Vietnam War', 'It isolated the US from its allies during the Vietnam War', 'It led to the immediate end of the Vietnam War']\",\"question\":\"What was the significance of the Gulf of Tonkin resolution?\",\"src\":\"ori_mmlu-us_foreign_policy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}