{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"7e572bd7-58c0-5025-9547-716bf52ef70a","task_key":"formal~5ffallacies--main--7e572bd7-58c0-5025-9547-716bf52ef70a","task_revision_id":"1","upstream_id":"","short_description":"\"Some football fans admire various clubs, others love only a single team. But…","config":"formal_fallacies","split":"main","body":"{\"input\":\"\\\"Some football fans admire various clubs, others love only a single team. But who is a fan of whom precisely? The following argument pertains to this question: First, every expert of PFC Ludogorets 1945 is an ex-fan of KF Skënderbeu. Second, every expert of PFC Ludogorets 1945 is an opponent to OGC Nice. Third, some fan of Burnley FC is an opponent to OGC Nice and an ex-fan of KF Skënderbeu. It follows that some fan of Burnley FC is an expert of PFC Ludogorets 1945.\\\"\\nIs the argument, given the explicitly stated premises, deductively valid or invalid?\\nOptions:\\n- valid \\n- invalid\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}