{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mmmu-pro","formal_name":"MMMU-Pro","introduction":"多様な学問分野の画像とテキストを組み合わせ、マルチモーダル理解を評価するベンチマークです。公式データカードでは各設定に1,730問があり、4択・10択の標準設定と画像のみの設定を区別します。\n\nMMMU-Pro evaluates multimodal understanding across academic disciplines using images and text. The official card lists 1,730 questions per setting, separating four-option and ten-option standard settings from the vision setting.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://mmmu-benchmark.github.io/","indexing_mode":"noindex"},"task_id":"0b82da03-24fc-5813-981f-a013670f5cac","task_key":"standard~20~2810~20options~29--test--validation~5fManage~5f29","task_revision_id":"1","upstream_id":"validation_Manage_29","short_description":"Email #1: E-mail from Vice President of E.A.S.Y. Manufacturing to Legislative…","config":"standard (10 options)","split":"test","body":"{\"img_type\":\"['Plots and Charts']\",\"options\":\"['No', 'Insufficient information', 'Partially', 'Yes', 'Possibly']\",\"question\":\"Email #1: E-mail from Vice President of E.A.S.Y. Manufacturing to Legislative Consultant and Staff Researcher: The results of the last election are a concern for us. The candidates we supported were defeated in more than two-thirds of the races we followed. The result is a harder battle to get policies that benefit our interests through the legislature, but it's also a budget issue. We can't continue throwing money at races we can't win, no matter how agreeable the candidate's stand on our issues may be. The fact that they share our viewpoints doesn't help if they can't get elected. How can we determine where it makes sense to invest? E-mail #2: E-mail from Legislative Consultant to Vice President of E.A.S.Y. Manufacturing and Staff Researcher: The demographic breakdown of elections where we supported the unsuccessful candidate can tell us a great deal. We've won only in one demographic, and our margin there is smaller than the margins by which we lost the others. <image 1> E-mail #3: E-mail from Staff Researcher to Vice President of E.A.S.Y. Manufacturing and Legislative Consultant: The demographic groups in which we lost are on track to become a majority of the U.S. electorate this century, and they vote. Right now, they make up 37% of the overall population and 28% of voters in the last election. By midcentury, we can expect the Hispanic population to rise from 17% to about 29% of the U.S. population, and Asian-Americans, now about 5%, will be about 9%. Our one successful demographic is the one whose share is declining. Now 63% of the population, non-Hispanic whites will be less than half of the population by 2050. Consider the following statement. Does the information in the three sources support the inference as stated? Statement: The legislative consultant will actively lobby Asian-American legislators, because the candidates that E.A.S.Y. Manufacturing supported were more successful with Asian-Americans than with other minority groups.\",\"subject\":\"Manage\"}","display_format":"text","language":"","answer_status":"published","assets":[{"kind":"file","reference_id":"task:0b82da03-24fc-5813-981f-a013670f5cac:1:image_1.png","filename":"image_1.png","media_type":"image/png","alt":""}],"source_url":"https://mmmu-benchmark.github.io/","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}