{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmmu-pro","formal_name":"MMMU-Pro","introduction":"MMMU-Pro evaluates multimodal understanding across academic disciplines using images and text. The official card lists 1,730 questions per setting, separating four-option and ten-option standard settings from the vision setting.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://mmmu-benchmark.github.io/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"82929010-5479-54f8-99cd-ae3f1e196b7b","task_key":"standard~20~2810~20options~29--test--test~5fHistory~5f76","task_revision_id":"4","upstream_id":"test_History_76","short_description":"<image 1> <image 2> In contrast with the poem, the table suggests that one…","config":"standard (10 options)","split":"test","body":"{\"img_type\":\"['Tables']\",\"options\":\"['the influence of the United States', 'the strength of the naval forces', 'the morale and determination of the Allied soldiers', 'superior intelligence gathering techniques', 'ratio of territory to gross domestic product', 'wise and strategic leadership', 'overall population and industrial production', 'the presence of France in the alliance', 'the economic stability of the Allies', 'the use of sophisticated weaponry']\",\"question\":\"<image 1> <image 2> In contrast with the poem, the table suggests that one important factor in the victory of the Allies over the Central Powers was\",\"subject\":\"History\"}","display_format":"text","language":"","answer_status":"published","assets":[{"kind":"file","reference_id":"task:82929010-5479-54f8-99cd-ae3f1e196b7b:4:image_1.png","filename":"image_1.png","media_type":"image/png","alt":""},{"kind":"file","reference_id":"task:82929010-5479-54f8-99cd-ae3f1e196b7b:4:image_2.png","filename":"image_2.png","media_type":"image/png","alt":""}],"source_url":"https://mmmu-benchmark.github.io/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}