{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmmu-pro","formal_name":"MMMU-Pro","introduction":"MMMU-Pro evaluates multimodal understanding across academic disciplines using images and text. The official card lists 1,730 questions per setting, separating four-option and ten-option standard settings from the vision setting.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://mmmu-benchmark.github.io/","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4d114ebe-8790-536d-b2e1-6b97ce2c4012","task_key":"standard~20~2810~20options~29--test--validation~5fEconomics~5f28","task_revision_id":"4","upstream_id":"validation_Economics_28","short_description":"In case 1, would the worker choose to work or enjoy leisure time? <image 1>","config":"standard (10 options)","split":"test","body":"{\"img_type\":\"['Plots and Charts']\",\"options\":\"['Leisure time is 8 hours.', 'Leisure time is 14 hours.', 'Leisure time is 12 hours.', 'Leisure time is 20 hours.', 'Leisure time is 18 hours.', 'Leisure time is 22 hours.', 'Leisure time is 10 hours.', 'Leisure time is 16 hours.', 'Leisure time is 24 hours.']\",\"question\":\"In case 1, would the worker choose to work or enjoy leisure time? <image 1>\",\"subject\":\"Economics\"}","display_format":"text","language":"","answer_status":"published","assets":[{"kind":"file","reference_id":"task:4d114ebe-8790-536d-b2e1-6b97ce2c4012:4:image_1.png","filename":"image_1.png","media_type":"image/png","alt":""}],"source_url":"https://mmmu-benchmark.github.io/","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}