{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"a02fedf5-617c-561f-bac8-1b41ffbd0e77","task_key":"salient~5ftranslation~5ferror~5fdetection--main--a02fedf5-617c-561f-bac8-1b41ffbd0e77","task_revision_id":"1","upstream_id":"","short_description":"The following translations from German to English contain a particular error.…","config":"salient_translation_error_detection","split":"main","body":"{\"input\":\"The following translations from German to English contain a particular error. That error will be one of the following types: Named Entities: An entity (names, places, locations, etc.) is changed to a different entity. Numerical Values: Numerical values (ordinals or cardinals), dates, and/or units are changed. Modifiers or Adjectives: The modifiers and adjectives pertaining to a noun are changed. Negation or Antonyms: Introduce or remove a negation or change comparatives to their antonyms. Facts: Trivial factual errors not pertaining to the above classes are introduced in the translations. Dropped Content: A significant clause in the translation is removed. Please identify that error.  Source: Die unvollständige Liste der Baudenkmale in Barsinghausen enthält Baudenkmale der Barsinghausener Kernstadt sowie der Ortsteile Bantorf, Barrigsen, Eckerde, Egestorf, Göxe, Großgoltern, Groß Munzel, Hohenbostel, Holtensen, Kirchdorf, Landringhausen, Langreder, Nordgoltern, Ostermunzel, Stemmen, Wichtringhausen und Winninghausen.\\nTranslation: The complete list of architectural monuments in Barsinghausen contains architectural monuments of the Barsinghausen core town as well as the districts bantorf, Barrigsen, Eckerde, Egestorf, Göxe, Großgoltern, Groß Munzel, Hohenbostel, Holtensen, Kirchdorf, Landringhausen, Langreder, Nordgoltern, Ostermunzel, Stemmen, Wichtringhausen and Winninghausen.\\nThe translation contains an error pertaining to\\nOptions:\\n(A) Modifiers or Adjectives\\n(B) Numerical Values\\n(C) Negation or Antonyms\\n(D) Named Entities\\n(E) Dropped Content\\n(F) Facts\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}