{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"mgsm","formal_name":"MGSM","introduction":"MGSM is a human translation of 250 GSM8K problems into eleven languages, measuring whether reasoning survives the change of language. The gap against English is the multilingual signal.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-research/url-nlp","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"fced20de-4b2f-5450-a464-a6c39ed54545","task_key":"sw--test--fced20de-4b2f-5450-a464-a6c39ed54545","task_revision_id":"1","upstream_id":"","short_description":"Sue anaishi katika mtaa wenye burudani. Wikendi moja, majirani waliamua…","config":"sw","split":"test","body":"{\"question\":\"Sue anaishi katika mtaa wenye burudani. Wikendi moja, majirani waliamua kumfanyia Sue mzaha. Ijumaa asubuhi, majirani waliweka famingo 18 pinki wa plastiki kwenye bustani la mbele la Sue. Jumamosi asubuhi, majirani waliondoa thuluthi moja ya flamingo, wakawapaka rangi nyeupe, na kisha wakarudisha flamingo hawa waliopakwa kwenye bustani la mbele la Sue. Kisha, Jumapili asubuhi, waliongeza flamingo wengine 18 wa pinki wa plastiki kwenye mkusanyiko huo. Adhuhuri Jumapili, kulikuwa na flamingo ngapi wa pinki zaidi ya wa rangi nyeupe?\"}","display_format":"math","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-research/url-nlp","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}