{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"48b5443d-e153-59ef-8840-297c3d45a3ab","task_key":"formal~5ffallacies--main--48b5443d-e153-59ef-8840-297c3d45a3ab","task_revision_id":"1","upstream_id":"","short_description":"\"Consumer research aims at understanding whether users of some products also…","config":"formal_fallacies","split":"main","body":"{\"input\":\"\\\"Consumer research aims at understanding whether users of some products also tend to consume other ones, or not. The following argument seeks to clarify some such relations: First, Nick is a frequent consumer of Colgate-Palmolive soap. Second, being a frequent consumer of Burt's Bees soap or not being a frequent consumer of Colgate-Palmolive soap is sufficient for being an occasional purchaser of Redken shampoo. All this entails that it is not the case that Nick is an occasional purchaser of Redken shampoo.\\\"\\nIs the argument, given the explicitly stated premises, deductively valid or invalid?\\nOptions:\\n- valid \\n- invalid\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}