{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmmu-pro","formal_name":"MMMU-Pro","introduction":"MMMU-Pro evaluates multimodal understanding across academic disciplines using images and text. The official card lists 1,730 questions per setting, separating four-option and ten-option standard settings from the vision setting.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://mmmu-benchmark.github.io/","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"f8ad14ef-45cf-53c8-936e-da85aa7117f6","task_key":"standard~20~2810~20options~29--test--test~5fArchitecture~5fand~5fEngineering~5f146","task_revision_id":"3","upstream_id":"test_Architecture_and_Engineering_146","short_description":"Compute all of the interior angles for each of the figures shown.<image 1>","config":"standard (10 options)","split":"test","body":"{\"img_type\":\"['Diagrams']\",\"options\":\"['A=81°35\\\\'30\\\";B=194°50\\\\'06\\\";C=83°35\\\\'24\\\"', 'A=86°34\\\\'30\\\";B=194°50\\\\'06\\\";C=83°35\\\\'24\\\"', 'A=81°34\\\\'30\\\";B=194°50\\\\'06\\\";C=83°34\\\\'24\\\"', 'A=83°34\\\\'30\\\";B=194°50\\\\'06\\\";C=83°35\\\\'24\\\"', 'A=82°34\\\\'30\\\";B=194°50\\\\'06\\\";C=83°35\\\\'24\\\"', 'A=84°34\\\\'30\\\";B=194°50\\\\'06\\\";C=83°35\\\\'24\\\"', 'A=81°34\\\\'30\\\";B=195°50\\\\'06\\\";C=83°35\\\\'24\\\"', 'A=81°34\\\\'30\\\";B=194°50\\\\'04\\\";C=83°35\\\\'24\\\"', 'A=81°34\\\\'30\\\";B=194°50\\\\'06\\\";C=83°35\\\\'24\\\"', 'A=81°34\\\\'32\\\";B=194°50\\\\'06\\\";C=83°35\\\\'24\\\"']\",\"question\":\"Compute all of the interior angles for each of the figures shown.<image 1>\",\"subject\":\"Architecture_and_Engineering\"}","display_format":"text","language":"","answer_status":"published","assets":[{"kind":"file","reference_id":"task:f8ad14ef-45cf-53c8-936e-da85aa7117f6:3:image_1.png","filename":"image_1.png","media_type":"image/png","alt":""}],"source_url":"https://mmmu-benchmark.github.io/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}