{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"15d5bf48-d9f5-5fc3-a8d6-360774458cd4","task_key":"formal~5ffallacies--main--15d5bf48-d9f5-5fc3-a8d6-360774458cd4","task_revision_id":"1","upstream_id":"","short_description":"\"Here comes a perfectly valid argument: To begin with, being an ingredient of…","config":"formal_fallacies","split":"main","body":"{\"input\":\"\\\"Here comes a perfectly valid argument: To begin with, being an ingredient of Spoiled is necessary for being an ingredient of SILK EYE PENCIL 08. Moreover, everything that is an ingredient of Careless Whisper is an ingredient of Spoiled, too. Finally, whatever is ingredient of Just To Clarify is at least one of these: an ingredient of Careless Whisper, an ingredient of SILK EYE PENCIL 08 or an ingredient of Blush Rush (Bare). We may conclude: To be an ingredient of Spoiled or an ingredient of Blush Rush (Bare) is necessary for being an ingredient of Just To Clarify.\\\"\\nIs the argument, given the explicitly stated premises, deductively valid or invalid?\\nOptions:\\n- valid \\n- invalid\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}