{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"947a6fb9-dd0b-5417-ac87-7172abf4005c","task_key":"formal~5ffallacies--main--947a6fb9-dd0b-5417-ac87-7172abf4005c","task_revision_id":"1","upstream_id":"","short_description":"\"Is Titanium oxide an ingredient of my washing power? Which chemicals does my…","config":"formal_fallacies","split":"main","body":"{\"input\":\"\\\"Is Titanium oxide an ingredient of my washing power? Which chemicals does my perfume contain? It is really difficult to keep track of all chemicals one is regularly exposed to. The following argument seeks to clarify some such relations: To start with, some ingredient of The Eraser is an ingredient of Lip pencil. Now, everything that is not both an ingredient of Full-On Lipstick and an ingredient of Lollibalm is however an ingredient of Lip pencil. We may conclude: Some ingredient of The Eraser is not both an ingredient of Full-On Lipstick and an ingredient of Lollibalm.\\\"\\nIs the argument, given the explicitly stated premises, deductively valid or invalid?\\nOptions:\\n- valid \\n- invalid\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}