{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"05bff902-235c-54a8-9dc2-089ab8673cda","task_key":"formal~5ffallacies--main--05bff902-235c-54a8-9dc2-089ab8673cda","task_revision_id":"1","upstream_id":"","short_description":"\"Is Fred a fan of Liverpool? Are supporters of Real Madrid devotees of PSG? In…","config":"formal_fallacies","split":"main","body":"{\"input\":\"\\\"Is Fred a fan of Liverpool? Are supporters of Real Madrid devotees of PSG? In European football, it is sometimes difficult to keep track of the mutual admiration and dislike. The following argument seeks to clarify some such relations: To begin with, every backer of Athletic Club is an ex-supporter of KKS Lech Poznań. Moreover, whoever is neither a devotee of Everton FC nor an ex-fan of KV Oostende is a backer of Athletic Club or an expert of Leicester City FC.finally, every expert of Leicester City FC is an ex-supporter of KKS Lech Poznań. We may conclude that everyone who is not an ex-supporter of KKS Lech Poznań is a devotee of Everton FC or not an ex-fan of KV Oostende.\\\"\\nIs the argument, given the explicitly stated premises, deductively valid or invalid?\\nOptions:\\n- valid \\n- invalid\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}