{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"8a117c89-3756-5ffa-8d90-544ea57d8932","task_key":"formal~5ffallacies--main--8a117c89-3756-5ffa-8d90-544ea57d8932","task_revision_id":"1","upstream_id":"","short_description":"\"Here comes a perfectly valid argument: First, everything that is both an…","config":"formal_fallacies","split":"main","body":"{\"input\":\"\\\"Here comes a perfectly valid argument: First, everything that is both an ingredient of DAGGER TATTOO LINER and an ingredient of AMANDE MILK VEIL is not an ingredient of All Wrapped Up. Second, being an ingredient of All Wrapped Up is necessary for being an ingredient of Gloss cuivre nacre. Hence, whatever is not an ingredient of DAGGER TATTOO LINER and not an ingredient of AMANDE MILK VEIL is however an ingredient of Gloss cuivre nacre.\\\"\\nIs the argument, given the explicitly stated premises, deductively valid or invalid?\\nOptions:\\n- valid \\n- invalid\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}