{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"11dc269e-0ba6-5ce4-9358-ca0adb20ffd9","task_key":"formal~5ffallacies--main--11dc269e-0ba6-5ce4-9358-ca0adb20ffd9","task_revision_id":"1","upstream_id":"","short_description":"\"Here comes a perfectly valid argument: First, every ingredient of Polar…","config":"formal_fallacies","split":"main","body":"{\"input\":\"\\\"Here comes a perfectly valid argument: First, every ingredient of Polar Attraction is an ingredient of Reel Him In. Second, every ingredient of Polar Attraction is not an ingredient of Reel Him In or an ingredient of The Dark Side. It follows that being an ingredient of Polar Attraction is sufficient for being an ingredient of The Dark Side.\\\"\\nIs the argument, given the explicitly stated premises, deductively valid or invalid?\\nOptions:\\n- valid \\n- invalid\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}