{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"58da07c3-bd3e-5cf3-8872-5a4f98a64765","task_key":"formal~5ffallacies--main--58da07c3-bd3e-5cf3-8872-5a4f98a64765","task_revision_id":"1","upstream_id":"","short_description":"\"Here comes a perfectly valid argument: To begin with, whatever is both an…","config":"formal_fallacies","split":"main","body":"{\"input\":\"\\\"Here comes a perfectly valid argument: To begin with, whatever is both an ingredient of Immense Mascara and an ingredient of I Adore You is an ingredient of Crawford's Wine or an ingredient of Bioglitz Color 9N.moreover, being an ingredient of lip2cheek is necessary for being an ingredient of Bioglitz Color 9N. Finally, everything that is an ingredient of Crawford's Wine is an ingredient of lip2cheek, too. Therefore, whatever is an ingredient of Immense Mascara and an ingredient of I Adore You is also an ingredient of lip2cheek.\\\"\\nIs the argument, given the explicitly stated premises, deductively valid or invalid?\\nOptions:\\n- valid \\n- invalid\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}