{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mgsm","formal_name":"MGSM","introduction":"MGSM is a human translation of 250 GSM8K problems into eleven languages, measuring whether reasoning survives the change of language. The gap against English is the multilingual signal.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-research/url-nlp","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a4a2d6f3-ab79-5a44-87cb-05645dfaa5d7","task_key":"fr--test--a4a2d6f3-ab79-5a44-87cb-05645dfaa5d7","task_revision_id":"1","upstream_id":"","short_description":"Jean aime boire un verre d'eau au petit déjeuner, au déjeuner et au dîner. Pour…","config":"fr","split":"test","body":"{\"question\":\"Jean aime boire un verre d'eau au petit déjeuner, au déjeuner et au dîner. Pour finir, il en boit également un avant d'aller se coucher. Il suit cette routine tous les jours de la semaine. Le week-end par contre, il aime se détendre et boire un soda au dîner. Combien de verres d'eau Jean boit-il en une semaine ?\"}","display_format":"math","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-research/url-nlp","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}