{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"ff3af6da-b814-5071-a4bc-5a20e893adeb","task_key":"formal~5ffallacies--main--ff3af6da-b814-5071-a4bc-5a20e893adeb","task_revision_id":"1","upstream_id":"","short_description":"\"It is not always easy to grasp who is consuming which products. The following…","config":"formal_fallacies","split":"main","body":"{\"input\":\"\\\"It is not always easy to grasp who is consuming which products. The following argument pertains to this question: First of all, every occasional purchaser of Matrix shampoo is an occasional purchaser of Organix Hair shampoo. Next, Linda is an occasional purchaser of Organix Hair shampoo. It follows that Linda is an occasional purchaser of Matrix shampoo.\\\"\\nIs the argument, given the explicitly stated premises, deductively valid or invalid?\\nOptions:\\n- valid \\n- invalid\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}