{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"abf6c3fd-c77f-5eb9-9ff3-1d38f2f43f46","task_key":"bbeh~5flinguini--full--abf6c3fd-c77f-5eb9-9ff3-1d38f2f43f46","task_revision_id":"1","upstream_id":"","short_description":"Agbirigba is a cant, or secret language, used by around 30 speakers of the…","config":"bbeh_linguini","split":"full","body":"{\"input\":\"Agbirigba is a cant, or secret language, used by around 30 speakers of the Ogbakiri dialect of the Ikwerre language. Here are some words in the Ogbakiri dialect, along with their Agbirigba forms and translations:\\n\\nOgbakiri | Agbirigba | \\nńdá | ńtʊ́dá | father\\nbі́sі́ | túbі́tі́sі́ | poison\\nɔyɔ̀ | ɔ̀tɪ̀yɔ̀ | amicable gesture\\nḿbɔ́rɔ̂ | ńtʊ́bɔ́tʊ́rɔ̀ | dream\\nɔ́ʧɪ́ | ɔ́tɪʧɪ́ ́ | leg\\nòtṹ | òtṹtṹ | to look for\\nmі̃́nі̃́ | tі̃́mі̃́tі̃́nі̃́ | water\\nwẽ̀rũ̀ | tũ̀wẽ̀tũ̀rũ̀ | to have\\nɛ̀bɔɔ́ | ɛ̀tʊ́bɔ́ | two\\nàwâ | àtʊ́wà | to break\\nrі́mẽ́ | tі́rі́tṹmẽ́ | inside\\nmɔ̃́nɔ̃̂ | tʊ̃́mɔ̃́tʊ̃́nɔ̃̀ | oil\\nі́bèrè | і́tùbètі̀rè | belch\\náyã̂ | átɪ̃́yã̀ | eye\\nɔ́nʊ̃́ | ɔ́tʊ̃́nʊ̃́ | mouth\\nmɔ̃́yɔ̃̂ | tʊ̃́mɔ̃́tɪ̃́yɔ̃̀ | urine\\nŋ̀gàdã́ | ǹtʊ̀gàtʊ̃́dã́ | chair\\nɔ́mɪ̃̀rɪ̃̀mã̀ | ɔ́tʊ̃̀mɪ̃̀tɪ̃̀rɪ̃̀tʊ̃̀mã̀ | meat, flesh\\nàgɪ̀rɪ̀gɔ̀ | àtɪ̀gɪ̀tɪ̀rɪ̀tʊ̀gɔ̀ | defiance\\nwẽ́nẽ́ | tṹwẽ́tі̃́nẽ́ | siblinghood\\n\\nGive the Agbirigba forms of the following Ogbakiri words:\\nɲ́yẽ́ rɪ̀yã̀ | female\\n\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}