{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b91082a4-d71f-50ee-bd6b-5386eb84a4eb","task_key":"formal~5ffallacies--main--b91082a4-d71f-50ee-bd6b-5386eb84a4eb","task_revision_id":"1","upstream_id":"","short_description":"\"Is Titanium oxide an ingredient of my washing power? Which chemicals does my…","config":"formal_fallacies","split":"main","body":"{\"input\":\"\\\"Is Titanium oxide an ingredient of my washing power? Which chemicals does my perfume contain? It is really difficult to keep track of all chemicals one is regularly exposed to. The following argument seeks to clarify some such relations: Every ingredient of CUSTOM BLEND POWDER is an ingredient of Gold Dust or an ingredient of Parisian Chic. Whatever is an ingredient of Parisian Chic is not an ingredient of CUSTOM BLEND POWDER. In consequence, every ingredient of CUSTOM BLEND POWDER is an ingredient of Gold Dust.\\\"\\nIs the argument, given the explicitly stated premises, deductively valid or invalid?\\nOptions:\\n- valid \\n- invalid\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}