{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0549d472-46f9-51d5-955a-9abea7e1424c","task_key":"salient~5ftranslation~5ferror~5fdetection--main--0549d472-46f9-51d5-955a-9abea7e1424c","task_revision_id":"1","upstream_id":"","short_description":"The following translations from German to English contain a particular error.…","config":"salient_translation_error_detection","split":"main","body":"{\"input\":\"The following translations from German to English contain a particular error. That error will be one of the following types: Named Entities: An entity (names, places, locations, etc.) is changed to a different entity. Numerical Values: Numerical values (ordinals or cardinals), dates, and/or units are changed. Modifiers or Adjectives: The modifiers and adjectives pertaining to a noun are changed. Negation or Antonyms: Introduce or remove a negation or change comparatives to their antonyms. Facts: Trivial factual errors not pertaining to the above classes are introduced in the translations. Dropped Content: A significant clause in the translation is removed. Please identify that error.  Source: Wouter Jacob Bos ist ein niederländischer Politiker, ehemaliger Parteiführer der sozialdemokratischen Partij van de Arbeid und vom 22. Februar 2007 bis den 23. Februar 2010 stellvertretender Ministerpräsident und Finanzminister der Niederlande.\\nTranslation: Wouter Jacob Bos is a Dutch politician, former leader of the Social Democratic Party of the Party of Arbeid and deputy prime minister of the Netherlands from 23 February 2007 to 23 February 2010.\\nThe translation contains an error pertaining to\\nOptions:\\n(A) Modifiers or Adjectives\\n(B) Numerical Values\\n(C) Negation or Antonyms\\n(D) Named Entities\\n(E) Dropped Content\\n(F) Facts\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}