{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"257db02b-022a-5f50-b6a9-b52d72ae4a02","task_key":"formal~5ffallacies--main--257db02b-022a-5f50-b6a9-b52d72ae4a02","task_revision_id":"1","upstream_id":"","short_description":"\"It is not always easy to grasp who is consuming which products. The following…","config":"formal_fallacies","split":"main","body":"{\"input\":\"\\\"It is not always easy to grasp who is consuming which products. The following argument pertains to this question: To start with, every infrequent user of Proactiv soap is both a regular consumer of Softsoap soap and a loyal buyer of Pureology shampoo. Now, it is not the case that Jasper is a regular consumer of Softsoap soap. In consequence, it is not the case that Jasper is an infrequent user of Proactiv soap.\\\"\\nIs the argument, given the explicitly stated premises, deductively valid or invalid?\\nOptions:\\n- valid \\n- invalid\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}