{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"42939d1a-fcba-5936-89a7-3e5ae3e062aa","task_key":"formal~5ffallacies--main--42939d1a-fcba-5936-89a7-3e5ae3e062aa","task_revision_id":"1","upstream_id":"","short_description":"\"It is not always easy to grasp who is consuming which products. The following…","config":"formal_fallacies","split":"main","body":"{\"input\":\"\\\"It is not always easy to grasp who is consuming which products. The following argument pertains to this question: First, whoever is not a rare consumer of Lever soap is an infrequent user of TRESemmé shampoo. Second, there exists an infrequent user of TRESemmé shampoo who is not an occasional purchaser of Paul Mitchell soap or not an occasional purchaser of Zest soap. In consequence, some rare consumer of Lever soap is not both an occasional purchaser of Paul Mitchell soap and an occasional purchaser of Zest soap.\\\"\\nIs the argument, given the explicitly stated premises, deductively valid or invalid?\\nOptions:\\n- valid \\n- invalid\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}