{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"bbh","formal_name":"BIG-Bench Hard","introduction":"BIG-Bench Hard is the set of 23 BIG-Bench tasks on which models of the time scored below the average human rater. It is best known for showing how much chain-of-thought prompting changes the result.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","indexing_mode":"index","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[],"history":"","construction":"","limitations":"","saturation":"","scores":[]}},"task_id":"4880937b-6ea4-58a9-8205-471597d44716","task_key":"formal~5ffallacies--main--4880937b-6ea4-58a9-8205-471597d44716","task_revision_id":"1","upstream_id":"","short_description":"\"Here comes a perfectly valid argument: To begin with, being an ingredient of…","config":"formal_fallacies","split":"main","body":"{\"input\":\"\\\"Here comes a perfectly valid argument: To begin with, being an ingredient of Lacquer Aquadelic is necessary for being an ingredient of Fashion Week Chic. Moreover, no ingredient of Most Wanted that is an ingredient of Juicy Kitten is an ingredient of Lacquer Aquadelic. From this follows: Whatever is neither an ingredient of Juicy Kitten nor an ingredient of Most Wanted is an ingredient of Fashion Week Chic.\\\"\\nIs the argument, given the explicitly stated premises, deductively valid or invalid?\\nOptions:\\n- valid \\n- invalid\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://github.com/suzgunmirac/BIG-Bench-Hard","history":"initial import","indexing_mode":"index","subproblems":[],"grids":[]}