{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"d6109beb-0ad3-592d-a9f6-54800f007ac6","task_key":"formal~5ffallacies--main--d6109beb-0ad3-592d-a9f6-54800f007ac6","task_revision_id":"1","upstream_id":"","short_description":"\"Here comes a perfectly valid argument: First, being an ingredient of CND…","config":"formal_fallacies","split":"main","body":"{\"input\":\"\\\"Here comes a perfectly valid argument: First, being an ingredient of CND VINYLUX is necessary for being an ingredient of Liquid Lips Catnip. Second, every ingredient of TV Paint Stick is not an ingredient of HIGHLIGHTER DUO or not an ingredient of Liquid Lips Catnip. Third, being an ingredient of CND VINYLUX is necessary for being an ingredient of HIGHLIGHTER DUO. We may conclude: Being an ingredient of CND VINYLUX is necessary for being an ingredient of TV Paint Stick.\\\"\\nIs the argument, given the explicitly stated premises, deductively valid or invalid?\\nOptions:\\n- valid \\n- invalid\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}