{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmlu-pro","formal_name":"MMLU-Pro","introduction":"MMLU-Pro rebuilds MMLU with ten answer options instead of four and removes items that no longer separate models. Questions span fourteen academic and professional subjects.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"40bcb1a8-c859-5b55-beca-002394e6397e","task_key":"default--test--5280","task_revision_id":"1","upstream_id":"5280","short_description":"Why did Franklin D. Roosevelt initially favour an 'isolationist' stance on the…","config":"default","split":"test","body":"{\"category\":\"other\",\"options\":\"['All of the above', 'He wanted to focus on strengthening the economy after the Great Depression', 'He believed that German domination of Europe would be in US interests', \\\"He believed that the European conflicts were not America's concern\\\", 'He believed that the US could benefit from staying neutral and trading with both sides', 'He thought that America was not ready for another war', 'He was under pressure from the Congress to maintain neutrality', 'He was wary of the potential cost and casualties of war', 'He believed that internal reform rather than international negotiation was the key to economic regeneration of the US', 'He always favoured isolationism']\",\"question\":\"Why did Franklin D. Roosevelt initially favour an 'isolationist' stance on the part of the US during the 1930s?\",\"src\":\"ori_mmlu-us_foreign_policy\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/TIGER-Lab/MMLU-Pro","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}