{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mgsm","formal_name":"MGSM","introduction":"MGSM is a human translation of 250 GSM8K problems into eleven languages, measuring whether reasoning survives the change of language. The gap against English is the multilingual signal.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-research/url-nlp","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"06bf9d68-1a84-5b0f-a254-727d6289d81a","task_key":"ja--test--06bf9d68-1a84-5b0f-a254-727d6289d81a","task_revision_id":"1","upstream_id":"","short_description":"スーは愉快な地域に住んでいます。ある週末、近所の人たちがスーにいたずらをしようと決めました。金曜日の朝、近所の人は18匹のプラスチックのピンクフラミンゴを、スー…","config":"ja","split":"test","body":"{\"question\":\"スーは愉快な地域に住んでいます。ある週末、近所の人たちがスーにいたずらをしようと決めました。金曜日の朝、近所の人は18匹のプラスチックのピンクフラミンゴを、スーの自宅前の庭に設置しました。土曜日の朝、近所の人はフラミンゴの3分の1をどかし、白く塗り、新しく塗られた白いフラミンゴを、スーの自宅前の庭に戻しました。それから、日曜日の朝、彼らは別の18個のプラスチックのピンクフラミンゴをそのコレクションに加えました。日曜日の午後、プラスチックのピンクフラミンゴはプラスチックの白いフラミンゴより何個多いですか？\"}","display_format":"math","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-research/url-nlp","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}