{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"06a3b50a-15f8-5247-a55c-4899b941b55a","task_key":"formal~5ffallacies--main--06a3b50a-15f8-5247-a55c-4899b941b55a","task_revision_id":"1","upstream_id":"","short_description":"\"Here comes a perfectly valid argument: First, being an ingredient of Hope…","config":"formal_fallacies","split":"main","body":"{\"input\":\"\\\"Here comes a perfectly valid argument: First, being an ingredient of Hope Moisture Mask is sufficient for being an ingredient of Miracle Concealer. Second, nothing is neither an ingredient of All Day Body Wash nor an ingredient of Hope Moisture Mask. We may conclude: Being an ingredient of All Day Body Wash is sufficient for not being an ingredient of Miracle Concealer.\\\"\\nIs the argument, given the explicitly stated premises, deductively valid or invalid?\\nOptions:\\n- valid \\n- invalid\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}