{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mgsm","formal_name":"MGSM","introduction":"MGSM is a human translation of 250 GSM8K problems into eleven languages, measuring whether reasoning survives the change of language. The gap against English is the multilingual signal.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-research/url-nlp","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"3128c172-dabc-5736-ad70-9cf31ab1befa","task_key":"de--test--3128c172-dabc-5736-ad70-9cf31ab1befa","task_revision_id":"1","upstream_id":"","short_description":"Sue lebt in einer lustigen Nachbarschaft. An einem Wochenende beschließen ihre…","config":"de","split":"test","body":"{\"question\":\"Sue lebt in einer lustigen Nachbarschaft. An einem Wochenende beschließen ihre Nachbarn Sue einen Streich zu spielen. Am Freitagmorgen stellen die Nachbarn 18 rosa Plastik-Flamingos draußen in Sues Vorgarten. Am Samstagmorgen nahmen die Nachbarn einen Drittel der Flamingos wieder weg, strichen sie weiß und stellten diese frisch gestrichenen weißen Flamingos zurück in Sues Vorgarten. Am Sonntagmorgen fügten sie der Sammlung dann weitere 18 rosa Plastik-Flamingos hinzu. Wie viele rosa Plastik-Flamingos waren am Sonntagnachmittag mehr als weiße Plastik-Flamingos draußen?\"}","display_format":"math","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-research/url-nlp","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}