{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"b47282a2-98c3-579e-8c81-4a452584cedd","task_key":"formal~5ffallacies--main--b47282a2-98c3-579e-8c81-4a452584cedd","task_revision_id":"1","upstream_id":"","short_description":"\"Here comes a perfectly valid argument: To begin with, whatever is not an…","config":"formal_fallacies","split":"main","body":"{\"input\":\"\\\"Here comes a perfectly valid argument: To begin with, whatever is not an ingredient of MAVA-WHITE is an ingredient of PRO LONGLASH. Moreover, not being an ingredient of Brow Powder Duo is sufficient for not being an ingredient of PRO LONGLASH. We may conclude that being an ingredient of Brow Powder Duo is necessary for not being an ingredient of MAVA-WHITE.\\\"\\nIs the argument, given the explicitly stated premises, deductively valid or invalid?\\nOptions:\\n- valid \\n- invalid\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}