{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmmu-pro","formal_name":"MMMU-Pro","introduction":"MMMU-Pro evaluates multimodal understanding across academic disciplines using images and text. The official card lists 1,730 questions per setting, separating four-option and ten-option standard settings from the vision setting.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://mmmu-benchmark.github.io/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"1b125dff-d669-5844-be9b-8837cf03de11","task_key":"standard~20~2810~20options~29--test--validation~5fArt~5fTheory~5f17","task_revision_id":"4","upstream_id":"validation_Art_Theory_17","short_description":"<image 1> The architect's philosophy of building is expressed by the idea that","config":"standard (10 options)","split":"test","body":"{\"img_type\":\"['Photographs']\",\"options\":\"['the integration of technology into design is essential', 'architecture should be minimalist in nature', 'buildings should be designed with the surrounding environment in mind', 'a house is a machine for living', 'construction should be environmentally friendly', 'less is more', 'all buildings should be sustainable', 'the aesthetics of the building are secondary to its practicality', 'only the use of natural material would be permitted', 'structures should primarily serve functional purposes']\",\"question\":\"<image 1> The architect's philosophy of building is expressed by the idea that\",\"subject\":\"Art_Theory\"}","display_format":"text","language":"","answer_status":"published","assets":[{"kind":"file","reference_id":"task:1b125dff-d669-5844-be9b-8837cf03de11:4:image_1.png","filename":"image_1.png","media_type":"image/png","alt":""}],"source_url":"https://mmmu-benchmark.github.io/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}