{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbeh","formal_name":"BIG-Bench Extra Hard (BBEH)","introduction":"BIG-Bench Extra Hard evaluates language-model reasoning through challenging task categories. It offers full and mini distributions; this import contains the full distribution: 4,520 questions across all 23 task categories.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/google-deepmind/bbeh","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"111280b7-aa0a-53b5-ac8f-d67c42616e74","task_key":"bbeh~5flinguini--full--111280b7-aa0a-53b5-ac8f-d67c42616e74","task_revision_id":"1","upstream_id":"","short_description":"The plural of Catalan nouns is usually formed by adding the ending -s. But if…","config":"bbeh_linguini","split":"full","body":"{\"input\":\"The plural of Catalan nouns is usually formed by adding the ending -s. But if the noun ends in one of the letters s, x or ç, more complex rules apply.\\nHere are the singular and the plural forms of some Catalan nouns (in simplified spelling) and their English translations. Some forms are missing.\\n\\nSingular | Plural | Translation\\nel apèndix | els apèndixs | appendix\\nel bastaix | (1) | carrier\\nel troleibús | (2) | trolleybus\\nel cactus | els cactus | cactus\\nla càries | les càries | caries\\n(3) | les clos | meadow\\nel contumaç | els contumaços | rebel\\nla faç | les façs | face\\nel flux | els fluxos | stream\\nel gimnàs | els gimnasos | gym\\nla hèlix | les hèlixs | screw\\nel índex | els índexs | index\\nel iris | (4) | rainbow\\nel llaç | (5) | loop\\nel ònix | (6) | onyx\\nel pàncrees | els pàncrees | pancreas\\nel pedaç | els pedaços | patch\\nla pelvis | (7) | pelvis\\nel permís | els permisos | permission\\nel pis | (8) | flat, lodgings\\n(9) | els sequaços | (male) follower\\n(10) | les sequaçs | (female) follower\\nel sufix | els sufixos | suffix\\n(11) | els tastaolletes | frivolous person\\nel teix | els teixos | yew\\nla trencadís | les trencadís | piece of crockery\\nel vas | els vasos | vase\\nla xeix | (12) | (the letter) x\\n\\nFill in the masked element indicated with (12), assuming the (1), (2), ... elements are masked out elements.\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/google-deepmind/bbeh","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}