{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"33af111c-1ffb-50dd-a686-00dfd1304e7b","task_key":"formal~5ffallacies--main--33af111c-1ffb-50dd-a686-00dfd1304e7b","task_revision_id":"1","upstream_id":"","short_description":"\"Is Titanium oxide an ingredient of my washing power? Which chemicals does my…","config":"formal_fallacies","split":"main","body":"{\"input\":\"\\\"Is Titanium oxide an ingredient of my washing power? Which chemicals does my perfume contain? It is really difficult to keep track of all chemicals one is regularly exposed to. The following argument seeks to clarify some such relations: To begin with, something that is not an ingredient of Maypole Soap is an ingredient of Cupcake or an ingredient of OW COLOR PROTECTION. Moreover, whatever is not an ingredient of Anti-Redness Cream is an ingredient of Cupcake. Finally, nothing is neither an ingredient of Anti-Redness Cream nor an ingredient of OW COLOR PROTECTION. Hence, nothing is neither an ingredient of Anti-Redness Cream nor an ingredient of Maypole Soap.\\\"\\nIs the argument, given the explicitly stated premises, deductively valid or invalid?\\nOptions:\\n- valid \\n- invalid\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}