{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"6d0ff797-0cef-54d2-b5e1-088e3013a982","task_key":"formal~5ffallacies--main--6d0ff797-0cef-54d2-b5e1-088e3013a982","task_revision_id":"1","upstream_id":"","short_description":"\"Here comes a perfectly valid argument: Everything that is both an ingredient of…","config":"formal_fallacies","split":"main","body":"{\"input\":\"\\\"Here comes a perfectly valid argument: Everything that is both an ingredient of HONEY LIPS and an ingredient of Renewal Cream is an ingredient of Take A Bow or an ingredient of LashGrip Clear. Every ingredient of Take A Bow is not an ingredient of Renewal Cream and not an ingredient of HONEY LIPS. From this follows: Everything that is not both an ingredient of HONEY LIPS and an ingredient of Renewal Cream is an ingredient of LashGrip Clear.\\\"\\nIs the argument, given the explicitly stated premises, deductively valid or invalid?\\nOptions:\\n- valid \\n- invalid\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}