{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"89f1d3d8-1612-515c-a934-22d974581be4","task_key":"bbeh~5flinguini--full--89f1d3d8-1612-515c-a934-22d974581be4","task_revision_id":"1","upstream_id":"","short_description":"Below you see sentences in the Tzotzil language (in the dialect of San Lorenzo…","config":"bbeh_linguini","split":"full","body":"{\"input\":\"Below you see sentences in the Tzotzil language (in the dialect of San Lorenzo Zinacantán) and their English translations:\\n1. 'Oy 'ox 'ixim ta ana nax. | You had corn at home today.\\n2. Bu 'oy 'ox li Romin e 'ok'ob? | Where will Domingo be tomorrow?\\n3. Ch'abal 'ox chenek' ta jp'in po'ot. | Soon there will be no haricots in my pot.\\n4. Mi 'oy 'ox k'in ta Jobel 'ok'ob? | Will there be a party in San Cristobal tomorrow?\\n5. 'Oy chan-vun ta batz'i k'op ta Jobel. | There is a Tzotzil school in San Cristobal.\\n6. Mi 'oy sbatz'i chi'il li Xun e? | Does Juan have a real friend?\\n7. Muk' bu li Xunka e. | Juana is nowhere.\\n8. \\\"Oy 'ox jlekil na po'ot. | I will soon have a good house.\\n9. Mi 'oy 'ox chan-vun ta Jobel junabi? | Was there a school in San Cristobal last year?\\n10. Mi 'oy \\\"ixim ta p'in lavie? | Is there corn in his pot?\\n11. Ch'abal schenek' lavie. | He has no haricots today.\\n12. 'Oy 'ox lekil vob ta k'in lavie. | There will be good music at the party today.\\n13. K'usi 'oy \\\"ox ta achan-vun volje? | What did you have at school yesterday?\\n14. Bu 'oy 'ox k'op nax? | Where was the talk today?\\n15. Ch'abal 'ox schi'il li Romin e junabi. | Last year Domingo had no friend.\\n\\nTranslate into Tzotzil:\\nYou have a real house.\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}