{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"0c712477-4dc2-5e13-8cb3-7aaeceeb0249","task_key":"formal~5ffallacies--main--0c712477-4dc2-5e13-8cb3-7aaeceeb0249","task_revision_id":"1","upstream_id":"","short_description":"\"Some football fans admire various clubs, others love only a single team. But…","config":"formal_fallacies","split":"main","body":"{\"input\":\"\\\"Some football fans admire various clubs, others love only a single team. But who is a fan of whom precisely? The following argument pertains to this question: To start with, no admirer of CFR 1907 Cluj is an expert of Inverness Caledonian Thistle FC. Now, being an expert of Inverness Caledonian Thistle FC is necessary for being an ex-fan of FC Viitorul. It follows that being an admirer of CFR 1907 Cluj is necessary for not being an ex-fan of FC Viitorul.\\\"\\nIs the argument, given the explicitly stated premises, deductively valid or invalid?\\nOptions:\\n- valid \\n- invalid\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}