{"kind":"task","effective_mode":"full","benchmark":{"kind":"benchmark","effective_mode":"full","slug":"longbench-v2","formal_name":"LongBench v2","introduction":"LongBench v2 evaluates deep understanding and reasoning over long contexts through multiple-choice questions. Its official description lists 503 questions spanning tasks such as single-document and multi-document QA and code-repository understanding.","introduction_ja":"","introduction_en":"","category":"Category not supplied","task_count":null,"acquisition_status":"Acquisition status not supplied","official_url":"https://huggingface.co/datasets/zai-org/LongBench-v2","indexing_mode":"noindex","profile":{"resources":[],"task_format":"","scoring":"","metric":"","size":"","answer_access":"","license":"","citation":"","maintainer":"","released":"","why_hard":"","related":[]}},"task_id":"92006e5d-17d8-52a3-b20e-8f6ac3c231fd","task_key":"train--66f530ce821e116aacb32f09","task_revision_id":"3","upstream_id":"66f530ce821e116aacb32f09","short_description":"This codebase implements a method for generating stylized narrative images from…","config":"","split":"train","body":"{\"choice_A\":\"The process begins by using the BirefNet model to effectively extract the human subject from the portrait. Once the subject is isolated, the SDXL model takes over, utilizing user-written prompts to generate detailed and lifelike portrait images. Following this, OpenPose and ControlNet are employed to add stylistic enhancements, transforming the generated portraits into visually compelling and stylized artworks that capture the desired narrative essence.\",\"choice_B\":\"The process begins with the YOLO model, which is utilized to detect and locate the human figure within the portrait. Once identified, the human is extracted from the image for further processing. Next, Ipadapter and Pulid are employed to generate high-quality portrait images that maintain the essence of the original subject. Finally, a LoRA model is applied to add stylistic elements, transforming the generated portraits into uniquely stylized artworks that enhance the overall visual appeal and narrative depth.\",\"choice_C\":\"The process begins by extracting the face from the portrait, ensuring a clear focus on the subject's features. An image recognition model is then utilized to generate descriptive prompts that capture the essence of the face. Using these prompts, the Flux model generates four distinct portrait images, each showcasing different artistic interpretations of the original face. Next, reactor face-swapping is applied to seamlessly blend the facial features across the generated images, enhancing diversity and creativity. Finally, the SDXL and ControlNet models are employed to apply stylistic enhancements, transforming the final output into a series of visually striking and stylized portraits that convey a rich narrative and artistic flair.\",\"choice_D\":\"Interpret the portrait using a LLM multimodal model, use stable diffusion base 1.5 to generate portrait images with annotated prompts, and apply a cartoonish model to regenerate for stylization.\",\"context\":\"# 使用方法\\n在 /root/ComfyUI/MYROUTER 目录下执行以下命令：\\n```shell\\npython model_server.py\\n```\\n\\n# 目前已集成的功能\\n\\n## 1. product_photography: 产品拍摄\\n\\n## 2. product_enhancement: 给产品换背景\\n\\n\\n## 3. dress_try : 衣服试穿\\n\\n## 4. flux_generation: 文生图模型\\n\\n\\n\\n\\nfrom routers import swagger_monkey_patch, add_comfyui_directory_to_sys_path, import_custom_nodes\\n\\nadd_comfyui_directory_to_sys_path()\\nimport_custom_nodes()\\n\\nfrom fastapi import applications\\napplications.get_swagger_ui_html  = swagger_monkey_patch\\n\\nfrom fastapi import FastAPI\\nfrom fastapi.middleware.cors import CORSMiddleware \\napp = FastAPI() # 配置 CORS\\napp.add_middleware( CORSMiddleware, allow_origins=[\\\"*\\\"], # 允许的源 \\n                    allow_credentials=True, allow_methods=[\\\"*\\\"], # 允许的 HTTP 方法 \\n                    allow_headers=[\\\"*\\\"], # 允许的 HTTP 头 \\n                    )\\n\\nfrom routers import file_model_server, comicper_realvision,story_diffusion,style_transfer, user_model\\napp.include_router(file_model_server.router)\\napp.include_router(comicper_realvision.router)\\napp.include_router(story_diffusion.router)\\napp.include_router(style_transfer.router)\\napp.include_router(user_model.router)\\n\\n\\n# 之后可以使用\\n# app.include_router(\\n#     admin.router,\\n#     prefix=\\\"/admin\\\",\\n#     tags=[\\\"admin\\\"],\\n#     dependencies=[Depends(get_token_header)],\\n#     responses={418: {\\\"description\\\": \\\"I'm a teapot\\\"}},\\n# )\\n\\n\\n@app.get(\\\"/\\\")\\nasync def root():\\n    return {\\\"message\\\": \\\"Hello Bigger Applications!\\\"}\\n\\nif __name__ == '__main__':\\n    import uvicorn, argparse\\n    parser = argparse.ArgumentParser()\\n    parser.add_argument(\\\"--host\\\", default=\\\"127.0.0.1\\\", type=str)\\n    parser.add_argument(\\\"--port\\\", default=6006, type=int)\\n    uvicorn.run(app, host=\\\"127.0.0.1\\\", port=6006)\\n\\n\\nfrom routers import text2text\\n# person_img_description = \\\"a young, East Asian man with a short, close-cropped haircut, wearing a plain white T-shirt. The image is captured against a plain, light-colored background, providing minimal distractions and focusing the viewer's attention on the subject. His expression is neutral, with slightly parted lips and a calm,\\\"\\n# en_prompts =\\\"he wake up in the bed. he go to run in the park. he eat a hamburger in the restaurant. he play basketball in the playground.\\\"\\n# prompt = f\\\"person apearance: {person_img_description}. four actions: {en_prompts}. please generate four detailed prompts  without title containing the person appearance, action and environment\\\"\\n# prompt = text2text(prompt=prompt)\\n# print(f\\\"zhipu prompt: {prompt}\\\")\\n# prompt_ls = prompt.split(\\\"\\\\n\\\")\\n# print(len(prompt_ls))\\n# print(prompt_ls)\\n\\nen_prompts =\\\"he wake up in the bed. he go to run in the park. he eat a hamburger in the restaurant. he play basketball in the playground.\\\"\\nprompt = f\\\"{en_prompts}. please expand four prompts with action and environment details. separate each prompt with a new line\\\"\\nprompt = text2text(prompt=prompt)\\nprint(f\\\"zhipu prompt: {prompt}\\\")\\nprompt_ls = prompt.split(\\\"\\\\n\\\")\\nprint(len(prompt_ls))\\nprint(prompt_ls)\\n\\n# 阅读该网址 https://support.huaweicloud.com/sdk-python-devg-obs/obs_22_0400.html\\n# 安装sdk pip install esdk-obs-python --trusted-host pypi.org\\n\\nfrom obs import ObsClient\\nfrom obs import PutObjectHeader, GetObjectHeader\\nimport traceback\\nimport os\\n\\nak = \\\"IZ96BHBOQCLE3R4M8N2F\\\"\\nsk = \\\"YppiRBPqirMpgDw9hJuNrLhLy6hwiQylnWwXUJPg\\\"\\n\\n#  server填写Bucket对应的Endpoint, 这里以华北-北京四为例，其他地区请按实际情况填写。\\nserver = \\\"obs.cn-north-4.myhuaweicloud.com\\\"\\nobsClient = ObsClient(access_key_id=ak, secret_access_key=sk, server=server)\\n# 推荐通过环境变量获取AKSK，这里也可以使用其他外部引入方式传入，如果使用硬编码可能会存在泄露风险。\\n# 您可以登录访问管理控制台获取访问密钥AK/SK，获取方式请参见https://support.huaweicloud.com/usermanual-ca/ca_01_0003.html。\\n# ak = os.getenv(\\\"AccessKeyID\\\")\\n# sk = os.getenv(\\\"SecretAccessKey\\\")\\n\\n\\ndef obs_upload_file(file_path:str, dir:str):\\n\\n    # 创建obsClient实例\\n    # 如果使用临时AKSK和SecurityToken访问OBS，需要在创建实例时通过security_token参数指定securityToken值\\n    try:\\n        # ！！！！！！！！！！！注意，上传对象的附加头域，支持公开读，否则链接无法直接访问\\n        headers = PutObjectHeader(acl=\\\"public-read\\\")\\n        bucketName = \\\"zoomglass\\\"\\n        # 【可选】待上传对象的MIME类型\\n        # headers.contentType = \\\"image/png\\\"\\n        # 对象名，即上传后的文件名\\n        # objectKey = \\\"test/test.txt\\\"\\n        # 待上传文件/文件夹的完整路径，如aa/bb.txt，或aa/\\n        # file_path = \\\"./test.txt\\\"\\n        # 上传文件的自定义元数据\\n        metadata = {\\\"meta1\\\": \\\"value1\\\", \\\"meta2\\\": \\\"value2\\\"}\\n        # 文件上传\\n        objectKey = f\\\"models/{dir}/{os.path.basename(file_path)}\\\"\\n        resp = obsClient.putFile(bucketName, objectKey, file_path, metadata, headers)\\n        # 返回码为2xx时，接口调用成功，否则接口调用失败\\n        if resp.status < 300:\\n            print(\\\"Put File Succeeded\\\")\\n            return f\\\"https://zoomglass.obs.cn-north-4.myhuaweicloud.com/{objectKey}\\\"\\n            \\n        else:\\n            print(\\\"Put File Failed\\\", resp.status)\\n            print(\\\"requestId:\\\", resp.requestId)\\n            print(\\\"errorCode:\\\", resp.errorCode)\\n            print(\\\"errorMessage:\\\", resp.errorMessage)\\n    except:\\n        print(\\\"Put File Failed\\\")\\n        print(traceback.format_exc())\\n        raise Exception(\\\"obs Put File Failed\\\")\\n\\ndef obs_download_file(file_path:str, dir:str):\\n    bucketName = \\\"zoomglass\\\"\\n    objectKey = f\\\"models/{dir}/{os.path.basename(file_path)}\\\"\\n    downloadPath = file_path\\n    try:\\n        headers = GetObjectHeader()\\n        resp = obsClient.getObject(bucketName, objectKey, downloadPath, headers=headers)\\n        if resp.status < 300:\\n            print(\\\"Download File Succeeded\\\")\\n        else:\\n            print(\\\"Download File Failed\\\", resp.status)\\n            print(\\\"requestId:\\\", resp.requestId)\\n            print(\\\"errorCode:\\\", resp.errorCode)\\n            print(\\\"errorMessage:\\\", resp.errorMessage)\\n    except:\\n        print(\\\"Download File Failed\\\")\\n        print(traceback.format_exc())\\n        raise Exception(\\\"obs Download File Failed\\\")\\n\\n\\ndef get_path_from_url(url: str, split_by:str=\\\"story_diffusion\\\"):\\n    x = url.split(f\\\"{split_by}/\\\")[1]\\n    task_id = x.split(\\\"/\\\")[0]\\n    filename = x.split(\\\"/\\\")[1]\\n    return task_id, filename\\n\\n\\nimport os\\nimport shutil\\nimport random\\nfrom datetime import datetime\\nfrom models import obs_client\\nfrom fastapi import UploadFile\\n\\n\\n\\\"\\\"\\\"\\nserver_url = \\\"https://u447318-bf7d-c7ca5acc.bjc1.seetacloud.com:8443\\\"\\n之间的方法，用来将图片上传到本地服务器，从本地文件夹中查找\\ndef split_file_url(url: str):\\n    x = url.split(\\\"task_id=\\\")[1]\\n    task_id = x.split(\\\"&\\\")[0]\\n    filename = x.split(\\\"&\\\")[1].replace(\\\"filename=\\\", \\\"\\\")\\n    return task_id, filename\\n\\n\\ndef get_file_from_url(task_id: str, filename: str):\\n    print(task_id,filename)\\n    url_format = '{http}/file_model/get_file_from_url?task_id={task_id}&filename={filename}'\\n    return url_format.format(http=server_url, task_id=task_id, filename=filename)\\n\\\"\\\"\\\"\\n\\n\\ndef upload_file(file: UploadFile, current_model: str):\\n    task_id = datetime.now().strftime(\\\"%Y%m%d_%H%M%S_\\\")+str(random.randint(1,1000))\\n    save_path = f'/root/autodl-tmp/data_store/{current_model}/{task_id}/'\\n\\n    suffix = file.filename.split('.')[-1]\\n    os.makedirs(os.path.join(save_path), exist_ok=True)\\n    while True:\\n        time_suffix = datetime.now().strftime(\\\"%Y%m%d_%H%M%S\\\")\\n        filename = f\\\"{time_suffix}.{suffix}\\\"\\n        path = os.path.join(save_path, filename)\\n        if not os.path.exists(path):\\n            break\\n    with open(path, \\\"wb\\\") as buffer:\\n        shutil.copyfileobj(file.file, buffer)\\n    user_image_url = obs_client.obs_upload_file(path, f\\\"{current_model}/{task_id}\\\")\\n    return user_image_url\\n\\n\\n#!/usr/bin/env python\\n# -*- coding: utf-8 -*-\\nimport datetime\\nimport base64\\nimport hmac\\nimport json\\nfrom hashlib import sha256 as sha256\\nimport requests\\n\\npid = '81000745'\\nsecret_key = b'J/YToXmmxEvbB61BdTqIIrvHPkd7gQldNQqhhH5BvSw='\\nendpoint_host = 'isafe.ilivedata.com'\\nendpoint_path = '/api/v1/image/check'\\nendpoint_url = 'https://isafe.ilivedata.com/api/v1/image/check'\\n\\ndef check_image_safety(image, type):\\n    now_date = datetime.datetime.now(datetime.timezone.utc).strftime('%Y-%m-%dT%H:%M:%SZ')\\n\\n    params = {\\n        \\\"type\\\": type,\\n        \\\"image\\\": image\\n    }\\n\\n    query_body = json.dumps(params)\\n    parameter = \\\"POST\\\\n\\\"\\n    parameter += endpoint_host + \\\"\\\\n\\\"\\n    parameter += endpoint_path + '\\\\n'\\n    parameter += sha256(query_body.encode('utf-8')).hexdigest() + \\\"\\\\n\\\"\\n    parameter += \\\"X-AppId:\\\" + pid + \\\"\\\\n\\\"\\n    parameter += \\\"X-TimeStamp:\\\" + now_date\\n\\n    signature = base64.b64encode(\\n        hmac.new(secret_key, parameter.encode('utf-8'), digestmod=sha256).digest())\\n\\n    headers = {\\n        \\\"X-AppId\\\": pid,\\n        \\\"X-TimeStamp\\\": now_date,\\n        \\\"Content-type\\\": \\\"application/json\\\",\\n        \\\"Authorization\\\": signature,\\n        \\\"Host\\\": endpoint_host,\\n        \\\"Connection\\\": \\\"keep-alive\\\"\\n    }\\n\\n    # querystring = parse.urlencode(params)\\n    resp = requests.post(endpoint_url, query_body, headers=headers)\\n    print(resp.text)\\n    if resp.status_code != 200:\\n        raise Exception(\\\"图片审核调用失败：\\\", resp.text)\\n    return resp.json()\\n\\nif __name__ == '__main__':\\n    # ！！！两种方式\\n  \\n# 1. 上传base64\\n    # with open('yellow.png',\\\"rb\\\") as f:\\n    #     image_base64 = base64.b64encode(f.read()).decode('utf-8')\\n    #     response = check_image_safety(image_base64, 2)\\n    #     print(response)\\n\\n    #     if response[\\\"result\\\"] == 2:\\n    #         print(\\\"不安全内容\\\", response[\\\"imageSpams\\\"])\\n    #     elif response[\\\"result\\\"] == 1:\\n    #         print(\\\"建议审核\\\")\\n    #     else:\\n    #         print(\\\"图片正常，继续执行\\\")\\n      \\n# 2. 上传url\\n    image_url = \\\"https://zoomglass.obs.cn-north-4.myhuaweicloud.com/models/story_diffusion/20240926_013639813/20240926_013639.jpg\\\"\\n    response = check_image_safety(image_url, 1)\\n    print(response)\\n\\n    if response[\\\"result\\\"] == 2:\\n        print(\\\"不安全内容\\\", response[\\\"imageSpams\\\"])\\n    elif response[\\\"result\\\"] == 1:\\n        print(\\\"建议审核\\\")\\n    else:\\n        print(\\\"图片正常，继续执行\\\")\\n\\n\\nimport os\\nimport random\\nimport torch\\nimport json\\nimport shutil\\nfrom datetime import datetime\\nfrom fastapi import APIRouter\\n\\nfrom models import obs_client\\nfrom routers import get_value_at_index\\nfrom routers import ImageResponse\\nfrom routers import SAVE_PATH, OUTPUT_PATH,CURRENT_MODEL, logger\\n\\nfrom nodes import (\\n    LoraLoaderModelOnly,\\n    LoadImage,\\n    SaveImage,\\n    NODE_CLASS_MAPPINGS,\\n    CLIPTextEncode,\\n    RepeatLatentBatch,\\n    VAEDecode,\\n    VAEEncode,\\n    VAELoader,\\n    CheckpointLoaderSimple,\\n    CLIPVisionLoader,\\n    KSampler,\\n)\\n\\nwith torch.inference_mode():\\n    loadimage = LoadImage()\\n\\n    cliptextencode = CLIPTextEncode()\\n    \\n    checkpointloadersimple = CheckpointLoaderSimple()\\n    checkpointloadersimple_159 = checkpointloadersimple.load_checkpoint(\\n        ckpt_name=\\\"cartoonish_v1.safetensors\\\"\\n    )\\n\\n    imagescaletototalpixels = NODE_CLASS_MAPPINGS[\\\"ImageScaleToTotalPixels\\\"]()\\n    getimagesizeandcount = NODE_CLASS_MAPPINGS[\\\"GetImageSizeAndCount\\\"]()\\n    layermask_maskgrow = NODE_CLASS_MAPPINGS[\\\"LayerMask: MaskGrow\\\"]()\\n    invertmask = NODE_CLASS_MAPPINGS[\\\"InvertMask\\\"]()\\n    maskimage = NODE_CLASS_MAPPINGS[\\\"MaskImage\\\"]()\\n    image_batch = NODE_CLASS_MAPPINGS[\\\"Image Batch\\\"]()\\n    wd14taggerpysssss = NODE_CLASS_MAPPINGS[\\\"WD14Tagger|pysssss\\\"]()\\n    \\n    powerpaintcliploader = NODE_CLASS_MAPPINGS[\\\"PowerPaintCLIPLoader\\\"]()\\n    powerpaintcliploader_121 = powerpaintcliploader.ppclip_loading(\\n        base=\\\"clip_l.safetensors\\\", powerpaint=\\\"pytorch_model.bin\\\"\\n    )\\n    vaeloader = VAELoader()\\n    vaeloader_161 = vaeloader.load_vae(\\n        vae_name=\\\"vaeFtMse840000EmaPruned_vaeFtMse840k.safetensors\\\"\\n    )  \\n    vaeencode = VAEEncode()\\n\\n    ipadaptermodelloader = NODE_CLASS_MAPPINGS[\\\"IPAdapterModelLoader\\\"]()\\n    ipadaptermodelloader_176 = ipadaptermodelloader.load_ipadapter_model(\\n        ipadapter_file=\\\"ip-adapter-faceid-plusv2_sd15.bin\\\"\\n    )\\n\\n    clipvisionloader = CLIPVisionLoader()\\n    clipvisionloader_178 = clipvisionloader.load_clip(\\n        clip_name=\\\"ViT-H-14-laion2B-s32B-b79K.safetensors\\\"\\n    )\\n\\n    ipadapterinsightfaceloader = NODE_CLASS_MAPPINGS[\\\"IPAdapterInsightFaceLoader\\\"]()\\n    ipadapterinsightfaceloader_177 = ipadapterinsightfaceloader.load_insightface(\\n        provider=\\\"CUDA\\\", model_name=\\\"buffalo_l\\\"\\n    )\\n    saveimage = SaveImage()\\n\\n\\nrouter = APIRouter(\\n    prefix=\\\"/img2img\\\",\\n    tags=[\\\"cartoonish\\\"],\\n    dependencies=[],\\n    responses={404: {\\\"description\\\": \\\"Not found\\\"}},\\n)\\n@router.post(\\\"/cartoonish\\\")\\ndef comicper_realvision(image_urls: list[str]):\\n    with torch.inference_mode():\\n        try:\\n            try:\\n                load_origin_images = []\\n                global task_id\\n                for temp_image_url in image_urls:\\n                    task_id, filename = obs_client.get_path_from_url(url=temp_image_url, split_by=CURRENT_MODEL)\\n                    image_path = os.path.join(SAVE_PATH, CURRENT_MODEL, task_id,filename)\\n                    if not os.path.exists(image_path):\\n                        resp = obs_client.obs_download_file(file_path=image_path, dir=f\\\"{CURRENT_MODEL}/{task_id}\\\")\\n                    loadimage = LoadImage()\\n                    loadimage_1 = loadimage.load_image(image=image_path)\\n                    load_origin_images.append(loadimage_1)\\n                \\n                image_batch_63 = image_batch.image_batch(\\n                    image_1=get_value_at_index(load_origin_images[0], 0),\\n                    image_2=get_value_at_index(load_origin_images[1], 0),\\n                    image_3=get_value_at_index(load_origin_images[2], 0),\\n                    image_4=get_value_at_index(load_origin_images[3], 0),\\n                )\\n            except Exception as e:\\n                logger.error(f\\\"Error loading images: {e}\\\")\\n                raise Exception(f\\\"Error loading images: {e}\\\")\\n    \\n            imagescaletototalpixels_102 = imagescaletototalpixels.upscale(\\n                upscale_method=\\\"nearest-exact\\\",\\n                megapixels=0.4,\\n                image=get_value_at_index(image_batch_63, 0),\\n            )\\n            \\n            getimagesizeandcount_61 = getimagesizeandcount.getsize(\\n                image=get_value_at_index(imagescaletototalpixels_102, 0)\\n            )\\n\\n            cliptextencode_160 = cliptextencode.encode(\\n                text=\\\"3D,text, watermark,embedding:verybadimagenegative_v1.3, \\\",\\n                clip=get_value_at_index(checkpointloadersimple_159, 1),\\n            )\\n\\n            vaeencode_164 = vaeencode.encode(\\n                pixels=get_value_at_index(getimagesizeandcount_61, 0),\\n                vae=get_value_at_index(vaeloader_161, 0),\\n            )\\n\\n            try:\\n                wd14taggerpysssss_167 = wd14taggerpysssss.tag(\\n                    model=\\\"wd-v1-4-vit-tagger-v2\\\",\\n                    threshold=0.35,\\n                    character_threshold=0.85,\\n                    replace_underscore=True,\\n                    trailing_comma=True,\\n                    exclude_tags=\\\"\\\",\\n                    image=get_value_at_index(getimagesizeandcount_61, 0),\\n                )\\n                wd14taggerpysssss_text_list_167 = get_value_at_index(wd14taggerpysssss_167, 0)\\n                text_1 = \\\" \\\".join(wd14taggerpysssss_text_list_167)\\n                text_2 = \\\"Asian,portrait,colorful,Fashion, bust, art,solo, avatar, solid color simple background, best quality, high resolution, clear, masterpiece, ultra high resolution, 8k, quhanlora (viewers: 1.2)，jewelry, earrings, simple background, jacket, colorful background, Smiling, cool, big eyes,trendy,High gloss, smooth,\\\"\\n\\n                text_concatenate_168 = text_1 + \\\",\\\" + text_2\\n            except Exception as e:\\n                logger.error(f\\\"wd14_167 tagger Error: {e}\\\")\\n                raise Exception(f\\\"wd14_167 tagger Error: {e}\\\")\\n\\n            cliptextencode_166 = cliptextencode.encode(\\n                text=get_value_at_index(text_concatenate_168, 0),\\n                clip=get_value_at_index(checkpointloadersimple_159, 1),\\n            )\\n\\n            loraloadermodelonly = LoraLoaderModelOnly()\\n            loraloadermodelonly_175 = loraloadermodelonly.load_lora_model_only(\\n                lora_name=\\\"ip-adapter-faceid-plusv2_sd15_lora.safetensors\\\",\\n                strength_model=1,\\n                model=get_value_at_index(checkpointloadersimple_159, 0),\\n            )\\n            try:\\n                ipadapterfaceid = NODE_CLASS_MAPPINGS[\\\"IPAdapterFaceID\\\"]()\\n                ipadapterfaceid_173 = ipadapterfaceid.apply_ipadapter(\\n                    weight=0.9,\\n                    weight_faceidv2=1,\\n                    weight_type=\\\"linear\\\",\\n                    combine_embeds=\\\"concat\\\",\\n                    start_at=0,\\n                    end_at=1,\\n                    embeds_scaling=\\\"V only\\\",\\n                    model=get_value_at_index(loraloadermodelonly_175, 0),\\n                    ipadapter=get_value_at_index(ipadaptermodelloader_176, 0),\\n                    image=get_value_at_index(getimagesizeandcount_61, 0),\\n                    clip_vision=get_value_at_index(clipvisionloader_178, 0),\\n                    insightface=get_value_at_index(ipadapterinsightfaceloader_177, 0),\\n                )\\n            except Exception as e:\\n                logger.error(f\\\"ipadapterfaceid_173 Error: {e}\\\")\\n                raise Exception(f\\\"ipadapterfaceid_173 Error: {e}\\\")\\n\\n            repeatlatentbatch = RepeatLatentBatch()\\n            repeatlatentbatch_169 = repeatlatentbatch.repeat(\\n                amount=1, samples=get_value_at_index(vaeencode_164, 0)\\n            )\\n\\n            try:\\n                ksampler = KSampler()\\n                ksampler_162 = ksampler.sample(\\n                    seed=random.randint(1, 2**64),\\n                    steps=25,\\n                    cfg=8,\\n                    sampler_name=\\\"euler_ancestral\\\",\\n                    scheduler=\\\"normal\\\",\\n                    denoise=0.4,\\n                    model=get_value_at_index(ipadapterfaceid_173, 0),\\n                    positive=get_value_at_index(cliptextencode_166, 0),\\n                    negative=get_value_at_index(cliptextencode_160, 0),\\n                    latent_image=get_value_at_index(repeatlatentbatch_169, 0),\\n                )\\n            except Exception as e:\\n                logger.error(f\\\"ksampler_162 Error: {e}\\\")\\n                raise Exception(f\\\"ksampler_162 Error: {e}\\\")\\n\\n            vaedecode = VAEDecode()\\n            vaedecode_163 = vaedecode.decode(\\n                samples=get_value_at_index(ksampler_162, 0),\\n                vae=get_value_at_index(vaeloader_161, 0),\\n            )\\n\\n            vaeencode_171 = vaeencode.encode(\\n                pixels=get_value_at_index(vaedecode_163, 0),\\n                vae=get_value_at_index(vaeloader_161, 0),\\n            )\\n\\n            try:\\n                ksampler_170 = ksampler.sample(\\n                    seed=random.randint(1, 2**64),\\n                    steps=25,\\n                    cfg=8,\\n                    sampler_name=\\\"euler_ancestral\\\",\\n                    scheduler=\\\"normal\\\",\\n                    denoise=0.4,\\n                    model=get_value_at_index(checkpointloadersimple_159, 0),\\n                    positive=get_value_at_index(cliptextencode_166, 0),\\n                    negative=get_value_at_index(cliptextencode_160, 0),\\n                    latent_image=get_value_at_index(vaeencode_171, 0),\\n                )\\n            except Exception as e:\\n                logger.error(f\\\"ksampler_170 Error: {e}\\\")\\n                raise Exception(f\\\"ksampler_170 Error: {e}\\\")\\n\\n            vaedecode_172 = vaedecode.decode(\\n                samples=get_value_at_index(ksampler_170, 0),\\n                vae=get_value_at_index(vaeloader_161, 0),\\n            )\\n\\n            time_suffix = datetime.now().strftime(\\\"%Y%m%d_%H%M%S\\\")\\n            saveimage_180 = saveimage.save_images(\\n                filename_prefix=f\\\"comicper_realvision_{time_suffix}\\\",\\n                images=get_value_at_index(vaedecode_172, 0),\\n            )\\n            print(saveimage_180)\\n            result_image_urls = []\\n            for i in range(4):\\n                filename = saveimage_180['ui']['images'][i]['filename']\\n                temp_image_url = obs_client.obs_upload_file(os.path.join(OUTPUT_PATH, filename), dir=f\\\"{CURRENT_MODEL}/{task_id}\\\")\\n                shutil.move(os.path.join(OUTPUT_PATH, filename), os.path.join(SAVE_PATH, CURRENT_MODEL, task_id, filename))\\n                result_image_urls.append(temp_image_url)\\n            \\n            json_file = os.path.join(SAVE_PATH, CURRENT_MODEL, task_id, f\\\"real_comic_imgs_{time_suffix}.json\\\")\\n            with open(json_file, \\\"w\\\") as f:\\n                image_data = {\\n                        \\\"real_images\\\": image_urls,\\n                        \\\"result_images\\\": result_image_urls,\\n                    }\\n                json.dump(image_data, f, indent=4, ensure_ascii=False)\\n            \\n            image_info = obs_client.obs_upload_file(file_path=json_file, dir=f\\\"{CURRENT_MODEL}/{task_id}\\\")\\n            print(f\\\"Saved image info to {image_info}\\\")\\n            torch.cuda.empty_cache()\\n            return ImageResponse(code=200, message=\\\"success\\\", data=result_image_urls)\\n        except Exception as e:\\n            logger.error(f\\\"Error: {e}\\\")\\n            return ImageResponse(code=500, message=f\\\"Internal Server Error:{e}\\\")\\n\\n\\nif __name__ == \\\"__main__\\\":\\n    pass\\n\\n\\nimport os\\nimport random\\nimport torch\\nimport json\\nimport shutil\\nfrom datetime import datetime\\nfrom fastapi import APIRouter\\n\\nfrom models import obs_client\\nfrom routers import get_value_at_index\\nfrom routers import ImageResponse\\nfrom routers import SAVE_PATH, OUTPUT_PATH,CURRENT_MODEL, logger\\n\\nfrom nodes import (\\n    LoraLoaderModelOnly,\\n    LoadImage,\\n    SaveImage,\\n    NODE_CLASS_MAPPINGS,\\n    CLIPTextEncode,\\n    RepeatLatentBatch,\\n    VAEDecode,\\n    VAEEncode,\\n    VAELoader,\\n    CheckpointLoaderSimple,\\n    CLIPVisionLoader,\\n    KSampler,\\n)\\n\\nwith torch.inference_mode():\\n    loadimage = LoadImage()\\n\\n    brushnetloader = NODE_CLASS_MAPPINGS[\\\"BrushNetLoader\\\"]()\\n    brushnetloader_113 = brushnetloader.brushnet_loading(\\n        brushnet=\\\"diffusion_pytorch_model.safetensors\\\", dtype=\\\"float16\\\"\\n    )\\n    cliptextencode = CLIPTextEncode()\\n    \\n    checkpointloadersimple = CheckpointLoaderSimple()\\n    checkpointloadersimple_114 = checkpointloadersimple.load_checkpoint(\\n        ckpt_name=\\\"majicMIX realistic 麦橘写实_v7.safetensors\\\"\\n    )\\n    checkpointloadersimple_159 = checkpointloadersimple.load_checkpoint(\\n        ckpt_name=\\\"cartoonish_v1.safetensors\\\"\\n    )\\n\\n    imagescaletototalpixels = NODE_CLASS_MAPPINGS[\\\"ImageScaleToTotalPixels\\\"]()\\n    getimagesizeandcount = NODE_CLASS_MAPPINGS[\\\"GetImageSizeAndCount\\\"]()\\n    birefnet_hugo = NODE_CLASS_MAPPINGS[\\\"BiRefNet_Hugo\\\"]()\\n    layermask_maskgrow = NODE_CLASS_MAPPINGS[\\\"LayerMask: MaskGrow\\\"]()\\n    invertmask = NODE_CLASS_MAPPINGS[\\\"InvertMask\\\"]()\\n    maskimage = NODE_CLASS_MAPPINGS[\\\"MaskImage\\\"]()\\n    wd14taggerpysssss = NODE_CLASS_MAPPINGS[\\\"WD14Tagger|pysssss\\\"]()\\n    \\n    powerpaintcliploader = NODE_CLASS_MAPPINGS[\\\"PowerPaintCLIPLoader\\\"]()\\n    powerpaintcliploader_121 = powerpaintcliploader.ppclip_loading(\\n        base=\\\"clip_l.safetensors\\\", powerpaint=\\\"pytorch_model.bin\\\"\\n    )\\n    vaeloader = VAELoader()\\n    vaeloader_161 = vaeloader.load_vae(\\n        vae_name=\\\"vaeFtMse840000EmaPruned_vaeFtMse840k.safetensors\\\"\\n    )  \\n    vaeencode = VAEEncode()\\n\\n    ipadaptermodelloader = NODE_CLASS_MAPPINGS[\\\"IPAdapterModelLoader\\\"]()\\n    ipadaptermodelloader_176 = ipadaptermodelloader.load_ipadapter_model(\\n        ipadapter_file=\\\"ip-adapter-faceid-plusv2_sd15.bin\\\"\\n    )\\n\\n    clipvisionloader = CLIPVisionLoader()\\n    clipvisionloader_178 = clipvisionloader.load_clip(\\n        clip_name=\\\"ViT-H-14-laion2B-s32B-b79K.safetensors\\\"\\n    )\\n\\n    ipadapterinsightfaceloader = NODE_CLASS_MAPPINGS[\\\"IPAdapterInsightFaceLoader\\\"]()\\n    ipadapterinsightfaceloader_177 = ipadapterinsightfaceloader.load_insightface(\\n        provider=\\\"CUDA\\\", model_name=\\\"buffalo_l\\\"\\n    )\\n    powerpaint = NODE_CLASS_MAPPINGS[\\\"PowerPaint\\\"]()\\n    layerutility_imageblend = NODE_CLASS_MAPPINGS[\\\"LayerUtility: ImageBlend\\\"]()\\n    saveimage = SaveImage()\\n\\n\\nrouter = APIRouter(\\n    prefix=\\\"/img2img\\\",\\n    tags=[\\\"comicper_realvision\\\"],\\n    dependencies=[],\\n    responses={404: {\\\"description\\\": \\\"Not found\\\"}},\\n)\\n@router.get(\\\"/comicper_realvision\\\")\\ndef comicper_realvision(image_url: str):\\n    task_id, filename = obs_client.get_path_from_url(image_url, CURRENT_MODEL)\\n    image_path = os.path.join(SAVE_PATH, CURRENT_MODEL, task_id, filename)\\n    if not os.path.exists(image_path):\\n        resp = obs_client.obs_download_file(file_path=image_path, dir=f\\\"{CURRENT_MODEL}/{task_id}\\\")\\n    \\n    with torch.inference_mode():\\n        try:\\n            loadimage_63 = loadimage.load_image(image=image_path)\\n    \\n            imagescaletototalpixels_102 = imagescaletototalpixels.upscale(\\n                upscale_method=\\\"nearest-exact\\\",\\n                megapixels=0.4,\\n                image=get_value_at_index(loadimage_63, 0),\\n            )\\n            \\n            getimagesizeandcount_61 = getimagesizeandcount.getsize(\\n                image=get_value_at_index(imagescaletototalpixels_102, 0)\\n            )\\n            \\n            birefnet_hugo_123 = birefnet_hugo.background_remove(\\n                background_color_name=\\\"transparency\\\",\\n                device=\\\"auto\\\",\\n                image=get_value_at_index(getimagesizeandcount_61, 0),\\n            )\\n\\n            layermask_maskgrow_124 = layermask_maskgrow.mask_grow(\\n                invert_mask=False,\\n                grow=17,\\n                blur=12,\\n                mask=get_value_at_index(birefnet_hugo_123, 1),\\n            )\\n\\n            invertmask_126 = invertmask.invert(\\n                mask=get_value_at_index(layermask_maskgrow_124, 0)\\n            )\\n\\n            maskimage_127 = maskimage.main(\\n                image=get_value_at_index(getimagesizeandcount_61, 0),\\n                mask=get_value_at_index(invertmask_126, 0),\\n            )\\n            try:\\n                wd14taggerpysssss_125 = wd14taggerpysssss.tag(\\n                    model=\\\"wd-v1-4-moat-tagger-v2\\\",\\n                    threshold=0.35,\\n                    character_threshold=1,\\n                    replace_underscore=False,\\n                    trailing_comma=False,\\n                    exclude_tags=\\\"1girl, solo, short_hair, dress,standing, monochrome,1boy \\\",\\n                    image=get_value_at_index(maskimage_127, 0),\\n                )\\n                wd14taggerpysssss_text_list_125 = get_value_at_index(wd14taggerpysssss_125, 0)\\n                text_125 = \\\" \\\".join(wd14taggerpysssss_text_list_125)\\n            except Exception as e:\\n                logger.error(f\\\"wd14_125 tagger Error: {e}\\\")\\n                raise Exception(f\\\"wd14_125 tagger Error: {e}\\\")\\n            \\n            cliptextencode_115 = cliptextencode.encode(\\n                text= text_125,\\n                clip=get_value_at_index(checkpointloadersimple_114, 1),\\n            )\\n\\n            cliptextencode_116 = cliptextencode.encode(\\n                text=\\\"human\\\", clip=get_value_at_index(checkpointloadersimple_114, 1)\\n            )\\n\\n            cliptextencode_160 = cliptextencode.encode(\\n                text=\\\"3D,text, watermark,embedding:verybadimagenegative_v1.3, \\\",\\n                clip=get_value_at_index(checkpointloadersimple_159, 1),\\n            )\\n\\n            vaeencode_164 = vaeencode.encode(\\n                pixels=get_value_at_index(getimagesizeandcount_61, 0),\\n                vae=get_value_at_index(vaeloader_161, 0),\\n            )\\n\\n            try:\\n                wd14taggerpysssss_167 = wd14taggerpysssss.tag(\\n                    model=\\\"wd-v1-4-vit-tagger-v2\\\",\\n                    threshold=0.35,\\n                    character_threshold=0.85,\\n                    replace_underscore=True,\\n                    trailing_comma=True,\\n                    exclude_tags=\\\"\\\",\\n                    image=get_value_at_index(getimagesizeandcount_61, 0),\\n                )\\n                wd14taggerpysssss_text_list_167 = get_value_at_index(wd14taggerpysssss_167, 0)\\n                text_1 = \\\" \\\".join(wd14taggerpysssss_text_list_167)\\n                text_2 = \\\"Asian,portrait,colorful,Fashion, bust, art,solo, avatar, solid color simple background, best quality, high resolution, clear, masterpiece, ultra high resolution, 8k, quhanlora (viewers: 1.2)，jewelry, earrings, simple background, jacket, colorful background, Smiling, cool, big eyes,trendy,High gloss, smooth,\\\"\\n\\n                text_concatenate_168 = text_1 + \\\",\\\" + text_2\\n            except Exception as e:\\n                logger.error(f\\\"wd14_167 tagger Error: {e}\\\")\\n                raise Exception(f\\\"wd14_167 tagger Error: {e}\\\")\\n\\n            cliptextencode_166 = cliptextencode.encode(\\n                text=get_value_at_index(text_concatenate_168, 0),\\n                clip=get_value_at_index(checkpointloadersimple_159, 1),\\n            )\\n\\n            loraloadermodelonly = LoraLoaderModelOnly()\\n            loraloadermodelonly_175 = loraloadermodelonly.load_lora_model_only(\\n                lora_name=\\\"ip-adapter-faceid-plusv2_sd15_lora.safetensors\\\",\\n                strength_model=1,\\n                model=get_value_at_index(checkpointloadersimple_159, 0),\\n            )\\n            try:\\n                ipadapterfaceid = NODE_CLASS_MAPPINGS[\\\"IPAdapterFaceID\\\"]()\\n                ipadapterfaceid_173 = ipadapterfaceid.apply_ipadapter(\\n                    weight=0.9,\\n                    weight_faceidv2=1,\\n                    weight_type=\\\"linear\\\",\\n                    combine_embeds=\\\"concat\\\",\\n                    start_at=0,\\n                    end_at=1,\\n                    embeds_scaling=\\\"V only\\\",\\n                    model=get_value_at_index(loraloadermodelonly_175, 0),\\n                    ipadapter=get_value_at_index(ipadaptermodelloader_176, 0),\\n                    image=get_value_at_index(getimagesizeandcount_61, 0),\\n                    clip_vision=get_value_at_index(clipvisionloader_178, 0),\\n                    insightface=get_value_at_index(ipadapterinsightfaceloader_177, 0),\\n                )\\n            except Exception as e:\\n                logger.error(f\\\"ipadapterfaceid_173 Error: {e}\\\")\\n                raise Exception(f\\\"ipadapterfaceid_173 Error: {e}\\\")\\n\\n            repeatlatentbatch = RepeatLatentBatch()\\n            repeatlatentbatch_169 = repeatlatentbatch.repeat(\\n                amount=1, samples=get_value_at_index(vaeencode_164, 0)\\n            )\\n\\n            try:\\n                ksampler = KSampler()\\n                ksampler_162 = ksampler.sample(\\n                    seed=random.randint(1, 2**64),\\n                    steps=25,\\n                    cfg=8,\\n                    sampler_name=\\\"euler_ancestral\\\",\\n                    scheduler=\\\"normal\\\",\\n                    denoise=0.4,\\n                    model=get_value_at_index(ipadapterfaceid_173, 0),\\n                    positive=get_value_at_index(cliptextencode_166, 0),\\n                    negative=get_value_at_index(cliptextencode_160, 0),\\n                    latent_image=get_value_at_index(repeatlatentbatch_169, 0),\\n                )\\n            except Exception as e:\\n                logger.error(f\\\"ksampler_162 Error: {e}\\\")\\n                raise Exception(f\\\"ksampler_162 Error: {e}\\\")\\n\\n            vaedecode = VAEDecode()\\n            vaedecode_163 = vaedecode.decode(\\n                samples=get_value_at_index(ksampler_162, 0),\\n                vae=get_value_at_index(vaeloader_161, 0),\\n            )\\n\\n            vaeencode_171 = vaeencode.encode(\\n                pixels=get_value_at_index(vaedecode_163, 0),\\n                vae=get_value_at_index(vaeloader_161, 0),\\n            )\\n\\n            try:\\n                ksampler_170 = ksampler.sample(\\n                    seed=random.randint(1, 2**64),\\n                    steps=25,\\n                    cfg=8,\\n                    sampler_name=\\\"euler_ancestral\\\",\\n                    scheduler=\\\"normal\\\",\\n                    denoise=0.4,\\n                    model=get_value_at_index(checkpointloadersimple_159, 0),\\n                    positive=get_value_at_index(cliptextencode_166, 0),\\n                    negative=get_value_at_index(cliptextencode_160, 0),\\n                    latent_image=get_value_at_index(vaeencode_171, 0),\\n                )\\n            except Exception as e:\\n                logger.error(f\\\"ksampler_170 Error: {e}\\\")\\n                raise Exception(f\\\"ksampler_170 Error: {e}\\\")\\n\\n            vaedecode_172 = vaedecode.decode(\\n                samples=get_value_at_index(ksampler_170, 0),\\n                vae=get_value_at_index(vaeloader_161, 0),\\n            )\\n\\n            birefnet_hugo_66 = birefnet_hugo.background_remove(\\n                background_color_name=\\\"transparency\\\",\\n                device=\\\"auto\\\",\\n                image=get_value_at_index(vaedecode_172, 0),\\n            )\\n\\n            invertmask_88 = invertmask.invert(\\n                mask=get_value_at_index(birefnet_hugo_66, 1)\\n            )\\n\\n            try:\\n                powerpaint_120 = powerpaint.model_update(\\n                    fitting=1,\\n                    function=\\\"object removal\\\",\\n                    scale=1,\\n                    start_at=0,\\n                    end_at=9999,\\n                    save_memory=\\\"none\\\",\\n                    model=get_value_at_index(checkpointloadersimple_114, 0),\\n                    vae=get_value_at_index(checkpointloadersimple_114, 2),\\n                    image=get_value_at_index(getimagesizeandcount_61, 0),\\n                    mask=get_value_at_index(layermask_maskgrow_124, 0),\\n                    powerpaint=get_value_at_index(brushnetloader_113, 0),\\n                    clip=get_value_at_index(powerpaintcliploader_121, 0),\\n                    positive=get_value_at_index(cliptextencode_115, 0),\\n                    negative=get_value_at_index(cliptextencode_116, 0),\\n                )\\n\\n                ksampler_117 = ksampler.sample(\\n                    seed=random.randint(1, 2**64),\\n                    steps=20,\\n                    cfg=7.5,\\n                    sampler_name=\\\"euler\\\",\\n                    scheduler=\\\"normal\\\",\\n                    denoise=1,\\n                    model=get_value_at_index(powerpaint_120, 0),\\n                    positive=get_value_at_index(powerpaint_120, 1),\\n                    negative=get_value_at_index(powerpaint_120, 2),\\n                    latent_image=get_value_at_index(powerpaint_120, 3),\\n                )\\n            except Exception as e:\\n                logger.error(f\\\"powerpaint or ksampler_117 Error: {e}\\\")\\n                raise Exception(f\\\"powerpaint or ksampler_117 Error: {e}\\\")\\n\\n            vaedecode_118 = vaedecode.decode(\\n                samples=get_value_at_index(ksampler_117, 0),\\n                vae=get_value_at_index(checkpointloadersimple_114, 2),\\n            )\\n\\n            layerutility_imageblend_89 = layerutility_imageblend.image_blend(\\n                invert_mask=True,\\n                blend_mode=\\\"normal\\\",\\n                opacity=100,\\n                background_image=get_value_at_index(vaedecode_118, 0),\\n                layer_image=get_value_at_index(birefnet_hugo_66, 0),\\n                layer_mask=get_value_at_index(invertmask_88, 0),\\n            )\\n\\n            time_suffix = datetime.now().strftime(\\\"%Y%m%d_%H%M%S\\\")\\n            saveimage_180 = saveimage.save_images(\\n                filename_prefix=f\\\"comicper_realvision_{time_suffix}\\\",\\n                images=get_value_at_index(layerutility_imageblend_89, 0),\\n            )\\n            print(saveimage_180)\\n            filename = saveimage_180['ui']['images'][0]['filename']\\n            result_image_url = obs_client.obs_upload_file(file_path=os.path.join(OUTPUT_PATH, filename), dir=f\\\"{CURRENT_MODEL}/{task_id}\\\")\\n            shutil.move(os.path.join(OUTPUT_PATH, filename), os.path.join(SAVE_PATH,CURRENT_MODEL, task_id, filename))\\n\\n            json_file = os.path.join(SAVE_PATH, CURRENT_MODEL, task_id, f\\\"real_comic_imgs_{time_suffix}.json\\\")\\n            with open(json_file, \\\"w\\\") as f:\\n                image_data = {\\n                        \\\"real_image\\\": image_url,\\n                        \\\"result_image\\\": result_image_url,\\n                    }\\n                json.dump(image_data, f, indent=4, ensure_ascii=False)\\n\\n            image_info = obs_client.obs_upload_file(file_path=json_file, dir=f\\\"{CURRENT_MODEL}/{task_id}\\\")\\n            \\n            print(f\\\"Saved image info to {image_info}\\\")\\n            torch.cuda.empty_cache()\\n            return ImageResponse(code=200, message=\\\"success\\\", data=result_image_url)\\n        except Exception as e:\\n            logger.error(f\\\"Error: {e}\\\")\\n            return ImageResponse(code=500, message=f\\\"Internal Server Error:{e}\\\")\\n\\n\\nif __name__ == \\\"__main__\\\":\\n    pass\\n\\n\\nimport os \\nfrom routers import SAVE_PATH,ImageResponse\\nfrom fastapi import APIRouter, UploadFile, File\\n\\nrouter = APIRouter(prefix=\\\"/user\\\",\\n            tags=[\\\"user\\\"],\\n            dependencies=[],\\n            responses={404: {\\\"description\\\": \\\"Not found\\\"}}, )\\n\\n@router.get(\\\"/current_image_url\\\")\\ndef get_current_image_url():\\n    try:\\n        with open(os.path.join(SAVE_PATH,\\\"log.txt\\\"), \\\"r\\\") as f:\\n            url = f.read()\\n        return ImageResponse(code=200,message=\\\"success\\\",data=url)\\n    except Exception as e:\\n        return ImageResponse(code=500,message=\\\"error to find the newest url\\\",data=str(e))\\n\\nimport os\\nimport json\\nimport random\\nimport shutil\\nimport torch\\nfrom datetime import datetime\\nfrom fastapi import APIRouter\\n\\nfrom models import fileModel, obs_client\\nfrom routers import ImageResponse, get_value_at_index, trans_zh2en, text2text\\nfrom routers import SAVE_PATH, OUTPUT_PATH,CURRENT_MODEL, logger\\n# 原先用的风格转换方法\\n# from routers.InstantStyle.infer_style_controlnet import style_transfer\\n\\nfrom nodes import (\\n    VAEDecode,\\n    CLIPTextEncode,\\n    UNETLoader,\\n    DualCLIPLoader,\\n    NODE_CLASS_MAPPINGS,\\n    SaveImage,\\n    EmptyLatentImage,\\n    VAELoader,\\n    LoadImage,\\n    LoraLoader,\\n)\\n\\nwith torch.inference_mode():\\n    try:\\n        emptylatentimage = EmptyLatentImage()\\n        emptylatentimage_1 = emptylatentimage.generate(\\n            width=512, height=512, batch_size=1\\n        )\\n\\n        joy_caption_load = NODE_CLASS_MAPPINGS[\\\"Joy_caption_load\\\"]()\\n        joy_caption_load_144 = joy_caption_load.gen(\\n            model=\\\"unsloth/Meta-Llama-3.1-8B-bnb-4bit\\\"\\n        )\\n        \\n        faceanalysismodels = NODE_CLASS_MAPPINGS[\\\"FaceAnalysisModels\\\"]()\\n        faceanalysismodels_1 = faceanalysismodels.load_models(\\n            library=\\\"insightface\\\", provider=\\\"CUDA\\\"\\n        )\\n\\n        facesegmentation = NODE_CLASS_MAPPINGS[\\\"FaceSegmentation\\\"]()\\n\\n        dualcliploader = DualCLIPLoader()\\n        dualcliploader_61 = dualcliploader.load_clip(\\n            clip_name1=\\\"t5xxl_fp8_e4m3fn.safetensors\\\",\\n            clip_name2=\\\"clip_l.safetensors\\\",\\n            type=\\\"flux\\\",\\n        )\\n\\n        cliptextencode = CLIPTextEncode()\\n\\n        vaeloader = VAELoader()\\n        vaeloader_10 = vaeloader.load_vae(vae_name=\\\"ae.safetensors\\\")\\n\\n        unetloader = UNETLoader()\\n        unetloader_12 = unetloader.load_unet(\\n            unet_name=\\\"flux1-schnell-fp8-e4m3fn.safetensors\\\", weight_dtype=\\\"fp8_e4m3fn\\\"\\n        )\\n        loraloader = LoraLoader()\\n        loraloader_1 = loraloader.load_lora(\\n            lora_name=\\\"boreal-v2.safetensors\\\",\\n            strength_model=1,\\n            strength_clip=1,\\n            model=get_value_at_index(unetloader_12, 0),\\n            clip=get_value_at_index(dualcliploader_61, 0),\\n        )\\n\\n        ksamplerselect = NODE_CLASS_MAPPINGS[\\\"KSamplerSelect\\\"]()\\n        randomnoise = NODE_CLASS_MAPPINGS[\\\"RandomNoise\\\"]()\\n        # fluxpromptenhance = NODE_CLASS_MAPPINGS[\\\"FluxPromptEnhance\\\"]()\\n\\n        loadimage = LoadImage()\\n        joy_caption = NODE_CLASS_MAPPINGS[\\\"Joy_caption\\\"]()\\n        stringfunctionpysssss = NODE_CLASS_MAPPINGS[\\\"StringFunction|pysssss\\\"]()\\n\\n        basicguider = NODE_CLASS_MAPPINGS[\\\"BasicGuider\\\"]()\\n        basicscheduler = NODE_CLASS_MAPPINGS[\\\"BasicScheduler\\\"]()\\n        samplercustomadvanced = NODE_CLASS_MAPPINGS[\\\"SamplerCustomAdvanced\\\"]()\\n        latent_batch = NODE_CLASS_MAPPINGS[\\\"Latent Batch\\\"]()\\n        vaedecode = VAEDecode()\\n        reactorfaceswap = NODE_CLASS_MAPPINGS[\\\"ReActorFaceSwap\\\"]()\\n        showtextpysssss = NODE_CLASS_MAPPINGS[\\\"ShowText|pysssss\\\"]()\\n        saveimage = SaveImage()\\n\\n    except Exception as e:\\n        logger.error(f\\\"Model Loading Error: {e}\\\")\\n\\n\\nrouter = APIRouter(\\n    prefix=\\\"/img\\\",\\n    tags=[\\\"story_diffusion\\\"],\\n    dependencies=[],\\n    responses={404: {\\\"description\\\": \\\"Not found\\\"}},\\n)\\n\\n@router.get(\\\"/story_diffusion\\\")\\nasync def flux_story_diffusion(image_url:str, zh_prompts:str=\\\"她早上从床上醒来。|她去公园晨跑。|她在街上走路。|她在办公室工作。\\\") -> ImageResponse:\\n    task_id, filename = obs_client.get_path_from_url(image_url, CURRENT_MODEL)\\n    image_path = os.path.join(SAVE_PATH, CURRENT_MODEL, task_id, filename)\\n    if not os.path.exists(image_path):\\n        resp = obs_client.obs_download_file(file_path=image_path, dir=f\\\"{CURRENT_MODEL}/{task_id}\\\")\\n    en_prompts = trans_zh2en(zh_prompts.replace(\\\"|\\\", \\\" \\\"))\\n    with torch.inference_mode():\\n        try:\\n            loadimage_1 = loadimage.load_image(image=image_path)\\n            facesegmentation_1 = facesegmentation.segment(\\n                area=\\\"face\\\",\\n                grow=70,\\n                grow_tapered=False,\\n                blur=1,\\n                analysis_models=get_value_at_index(faceanalysismodels_1, 0),\\n                image=get_value_at_index(loadimage_1, 0),\\n            )\\n            try:\\n                joy_caption_1 = joy_caption.gen(\\n                    prompt=\\\"This is a high-resolution image of\\\",\\n                    max_new_tokens=60,\\n                    temperature=1,\\n                    cache=True,\\n                    joy_pipeline=get_value_at_index(joy_caption_load_144, 0),\\n                    image=get_value_at_index(facesegmentation_1, 3),\\n                )\\n                person_img_description = get_value_at_index(joy_caption_1, 0)\\n                print(f\\\"[person_img_description]: {person_img_description}\\\")\\n                # prompt = f\\\"person apearance: {person_img_description}. four actions: {en_prompts}. please generate four prompts containing the person appearance, action and environment without title\\\"\\n                prompt = f\\\"{en_prompts}. please expand four prompts with action and environment details. separate each prompt with a new line\\\"\\n                prompt = text2text(prompt=prompt)\\n                prompt_ls = prompt.split(\\\"\\\\n\\\")\\n                prompt_ls = [prom for prom in prompt_ls if prom]\\n                image_ask =\\\"This is a photograph taken from a distance, showing the upper half of the frame.\\\"\\n                torch.cuda.empty_cache()\\n                samplercustomadvanced_1_list = []\\n            except Exception as e:\\n                logger.error(f\\\"Prompt handling Error: {e}\\\")\\n                raise Exception(f\\\"Prompt handling Error: {e}\\\")\\n            \\n            randomnoise_1 = randomnoise.get_noise(noise_seed=random.randint(1, 2**64))\\n            ksamplerselect_1 = ksamplerselect.get_sampler(sampler_name=\\\"euler\\\")\\n\\n            for prompt in prompt_ls:\\n                new_prompt = prompt + person_img_description + image_ask\\n                print(\\\"new prompt:\\\",new_prompt)\\n                cliptextencode_1 = cliptextencode.encode(\\n                    text = new_prompt,\\n                    clip=get_value_at_index(loraloader_1, 1),\\n                )\\n                basicscheduler_1 = basicscheduler.get_sigmas(\\n                    scheduler=\\\"simple\\\",\\n                    steps=3,\\n                    denoise=1,\\n                    model=get_value_at_index(loraloader_1, 0),\\n                )\\n\\n                basicguider_1 = basicguider.get_guider(\\n                    model=get_value_at_index(loraloader_1, 0),\\n                    conditioning=get_value_at_index(cliptextencode_1, 0),\\n                )\\n                try:\\n                    samplercustomadvanced_1 = samplercustomadvanced.sample(\\n                        noise=get_value_at_index(randomnoise_1, 0),\\n                        guider=get_value_at_index(basicguider_1, 0),\\n                        sampler=get_value_at_index(ksamplerselect_1, 0),\\n                        sigmas=get_value_at_index(basicscheduler_1, 0),\\n                        latent_image=get_value_at_index(emptylatentimage_1, 0),\\n                    )\\n                    samplercustomadvanced_1_list.append(get_value_at_index(samplercustomadvanced_1,0))\\n                except Exception as e:\\n                    logger.error(f\\\"Sampling Error: {e}\\\")\\n                    raise Exception(f\\\"Sampling Error: {e}\\\")\\n\\n            torch.cuda.empty_cache()\\n            latent_batch_1 = latent_batch.latent_batch(\\n                latent_1=samplercustomadvanced_1_list[0],\\n                latent_2=samplercustomadvanced_1_list[1],\\n                latent_3=samplercustomadvanced_1_list[2],\\n                latent_4=samplercustomadvanced_1_list[3],\\n            )\\n\\n            vaedecode_1 = vaedecode.decode(\\n                samples=get_value_at_index(latent_batch_1, 0),\\n                vae=get_value_at_index(vaeloader_10, 0),\\n            )\\n            try:\\n                reactorfaceswap_1 = reactorfaceswap.execute(\\n                    enabled=True,\\n                    swap_model=\\\"inswapper_128.onnx\\\",\\n                    facedetection=\\\"retinaface_resnet50\\\",\\n                    face_restore_model=\\\"GFPGANv1.4.pth\\\",\\n                    face_restore_visibility=1,\\n                    codeformer_weight=0.5,\\n                    detect_gender_input=\\\"no\\\",\\n                    detect_gender_source=\\\"no\\\",\\n                    input_faces_index=\\\"0\\\",\\n                    source_faces_index=\\\"0\\\",\\n                    console_log_level=1,\\n                    input_image=get_value_at_index(vaedecode_1, 0),\\n                    source_image=get_value_at_index(loadimage_1, 0),\\n                )\\n            except Exception as e:\\n                logger.error(f\\\"Reactor Face Swap Error: {e}\\\")\\n                raise Exception(f\\\"Reactor Face Swap Error: {e}\\\")\\n            \\n            time_suffix = datetime.now().strftime(\\\"%Y%m%d_%H%M%S\\\")\\n            saveimage_1 = saveimage.save_images(\\n                filename_prefix=f\\\"real_vision_{time_suffix}\\\",\\n                images=get_value_at_index(reactorfaceswap_1, 0),\\n            )\\n            print(f\\\"saveimage_1: {saveimage_1}\\\")\\n            real_image_urls = []\\n            for i in range(4):\\n                filename = saveimage_1['ui']['images'][i]['filename']\\n                temp_image_url = obs_client.obs_upload_file(file_path=os.path.join(OUTPUT_PATH, filename), dir = f\\\"{CURRENT_MODEL}/{task_id}\\\")\\n                shutil.move(os.path.join(OUTPUT_PATH, filename), os.path.join(SAVE_PATH, CURRENT_MODEL, task_id, filename))\\n                real_image_urls.append(temp_image_url)\\n            \\n            json_file = os.path.join(SAVE_PATH, CURRENT_MODEL, task_id, f\\\"images_prompts_{time_suffix}.json\\\")\\n            with open(json_file, \\\"w\\\") as f:\\n                image_data = {\\n                        \\\"person_img_description\\\": person_img_description,\\n                        \\\"chinese_prompts\\\": zh_prompts,\\n                        \\\"prompts\\\": prompt_ls,\\n                        \\\"user_image\\\": image_url,\\n                        \\\"images_without_styles\\\": real_image_urls, \\n                    }\\n                json.dump(image_data, f, indent=4, ensure_ascii=False)\\n            image_info = obs_client.obs_upload_file(file_path=json_file, dir=f\\\"{CURRENT_MODEL}/{task_id}\\\")\\n            print(f\\\"Saved image info to {image_info}\\\")\\n\\n            torch.cuda.empty_cache()\\n            return ImageResponse(code=200, message=\\\"success\\\", data=real_image_urls)\\n        \\n        except Exception as e:\\n            logger.error(f\\\"Error: {e}\\\")\\n            return ImageResponse(code=500, message=f\\\"Internal Server Error:{e}\\\")\\n\\nif __name__ == \\\"__main__\\\":\\n    zh_prompts = \\\"她早上从床上醒来。|她去公园晨跑。|她在街上走路。|她在办公室工作。\\\"\\n    flux_story_diffusion(prompts=zh_prompts)\\n\\n\\nimport os\\nimport shutil\\nimport uuid\\n\\nfrom fastapi import APIRouter, UploadFile, File, requests\\nfrom fastapi.responses import FileResponse\\nfrom pydantic import BaseModel\\nfrom starlette.responses import StreamingResponse\\n\\nimport models.fileModel as fileModel\\nfrom routers import ImageResponse, CURRENT_MODEL,SAVE_PATH\\n\\nrouter = APIRouter(prefix=\\\"/file_model\\\",\\n                   tags=[\\\"file_model\\\"],\\n                   dependencies=[],\\n                   responses={404: {\\\"description\\\": \\\"Not found\\\"}}, )\\n\\n@router.post(\\\"/uploadFile\\\")\\nasync def updateFile(file: UploadFile):\\n    url = fileModel.upload_file(file, current_model=CURRENT_MODEL)\\n    with open(os.path.join(SAVE_PATH,\\\"log.txt\\\"), \\\"w\\\") as f:\\n        f.write(url)\\n    response = ImageResponse(code=200, data=url, message=\\\"success\\\")\\n    return response\\n\\n\\nimport os\\nimport random\\nimport json\\nimport shutil\\nimport torch\\nfrom datetime import datetime\\nfrom fastapi import APIRouter\\n\\nfrom models import fileModel, obs_client\\nfrom routers import ImageResponse, get_value_at_index\\nfrom routers import OUTPUT_PATH, SAVE_PATH,CURRENT_MODEL, style_image_list, logger, image_style\\n\\nfrom nodes import (\\n    KSampler,\\n    CLIPTextEncode,\\n    ControlNetApply,\\n    NODE_CLASS_MAPPINGS,\\n    VAEDecode,\\n    LoadImage,\\n    EmptyLatentImage,\\n    ControlNetLoader,\\n    CheckpointLoaderSimple,\\n    SaveImage,\\n)\\n\\nwith torch.inference_mode():\\n    try:\\n        loadimage = LoadImage()\\n\\n        checkpointloadersimple = CheckpointLoaderSimple()\\n        checkpointloadersimple_166 = checkpointloadersimple.load_checkpoint(\\n            ckpt_name=\\\"sd_xl_base_1.0.safetensors\\\"\\n        )\\n        cliptextencode = CLIPTextEncode()\\n\\n        controlnetloader = ControlNetLoader()\\n        controlnetloader_174 = controlnetloader.load_controlnet(\\n            control_net_name=\\\"diffusers-controlnet-canny-sdxl-1.safetensors\\\"\\n        )\\n\\n        controlnetloader_175 = controlnetloader.load_controlnet(\\n            control_net_name=\\\"diffusers-controlnet-depth-sdxl-1.safetensors\\\"\\n        )\\n\\n        ipadapterunifiedloader = NODE_CLASS_MAPPINGS[\\\"IPAdapterUnifiedLoader\\\"]()\\n        imagebatchmultiple = NODE_CLASS_MAPPINGS[\\\"ImageBatchMultiple+\\\"]()\\n        ipadapteradvanced = NODE_CLASS_MAPPINGS[\\\"IPAdapterAdvanced\\\"]()\\n        image_batch = NODE_CLASS_MAPPINGS[\\\"Image Batch\\\"]()\\n        cannyedgepreprocessor = NODE_CLASS_MAPPINGS[\\\"CannyEdgePreprocessor\\\"]()\\n        controlnetapply = ControlNetApply()\\n        aio_preprocessor = NODE_CLASS_MAPPINGS[\\\"AIO_Preprocessor\\\"]()\\n        emptylatentimage = EmptyLatentImage()\\n        ksampler = KSampler()\\n        vaedecode = VAEDecode()\\n        imageluminancedetector = NODE_CLASS_MAPPINGS[\\\"ImageLuminanceDetector\\\"]()\\n        saveimage = SaveImage()\\n    except Exception as e:\\n        logger.error(f\\\"Error loading nodes: {e}\\\")\\n\\n\\n\\nrouter = APIRouter(\\n    prefix=\\\"/img\\\",\\n    tags=[\\\"style_transfer\\\"],\\n    dependencies=[],\\n    responses={404: {\\\"description\\\": \\\"Not found\\\"}},\\n)\\n\\n@router.post(\\\"/style_transfer\\\")\\nasync def style_transfer(image_urls:list[str], style:str=\\\"Line Art\\\")-> ImageResponse:\\n    with torch.inference_mode():\\n        try:\\n            try:\\n                load_origin_images = []\\n                global task_id\\n                for temp_image_url in image_urls:\\n                    task_id, filename = obs_client.get_path_from_url(url=temp_image_url, split_by=CURRENT_MODEL)\\n                    image_path = os.path.join(SAVE_PATH, CURRENT_MODEL, task_id,filename)\\n                    if not os.path.exists(image_path):\\n                        resp = obs_client.obs_download_file(file_path=image_path, dir=f\\\"{CURRENT_MODEL}/{task_id}\\\")\\n                    loadimage = LoadImage()\\n                    loadimage_1 = loadimage.load_image(image=image_path)\\n                    load_origin_images.append(loadimage_1)\\n\\n                load_style_images = []\\n                for image_name in style_image_list[style]:\\n                    loadimage = LoadImage()\\n                    loadimage_1 = loadimage.load_image(image=image_name)\\n                    load_style_images.append(loadimage_1)\\n            except Exception as e:\\n                logger.error(f\\\"Error loading images: {e}\\\")\\n                raise Exception(f\\\"Error loading images: {e}\\\")\\n\\n            style_description=image_style[style]\\n            negative_description=\\\"nsfw, lowres, bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality, normal quality, jpeg artifacts, signature, watermark, username, blurry, artist name\\\"\\n\\n            cliptextencode_169 = cliptextencode.encode(\\n                text=style_description,\\n                clip=get_value_at_index(checkpointloadersimple_166, 1),\\n            )\\n\\n            cliptextencode_170 = cliptextencode.encode(\\n                text=negative_description,\\n                clip=get_value_at_index(checkpointloadersimple_166, 1),\\n            )\\n\\n            ipadapterunifiedloader_38 = ipadapterunifiedloader.load_models(\\n                preset=\\\"PLUS (high strength)\\\",\\n                model=get_value_at_index(checkpointloadersimple_166, 0),\\n            )\\n\\n            imagebatchmultiple_45 = imagebatchmultiple.execute(\\n                method=\\\"nearest-exact\\\",\\n                image_1=get_value_at_index(load_style_images[0], 0),\\n                image_2=get_value_at_index(load_style_images[1], 0),\\n                image_3=get_value_at_index(load_style_images[2], 0),\\n                image_4=get_value_at_index(load_style_images[3], 0),\\n            )\\n\\n            ipadapteradvanced_37 = ipadapteradvanced.apply_ipadapter(\\n                weight=1,\\n                weight_type=\\\"style transfer precise\\\",\\n                combine_embeds=\\\"concat\\\",\\n                start_at=0,\\n                end_at=1,\\n                embeds_scaling=\\\"V only\\\",\\n                model=get_value_at_index(ipadapterunifiedloader_38, 0),\\n                ipadapter=get_value_at_index(ipadapterunifiedloader_38, 1),\\n                image=get_value_at_index(imagebatchmultiple_45, 0),\\n            )\\n\\n            image_batch_230 = image_batch.image_batch(\\n                image_1=get_value_at_index(load_origin_images[0], 0),\\n                image_2=get_value_at_index(load_origin_images[1], 0),\\n                image_3=get_value_at_index(load_origin_images[2], 0),\\n                image_4=get_value_at_index(load_origin_images[3], 0),\\n            )\\n\\n            cannyedgepreprocessor_177 = cannyedgepreprocessor.execute(\\n                low_threshold=50,\\n                high_threshold=200,\\n                resolution=512,\\n                image=get_value_at_index(image_batch_230, 0),\\n            )\\n\\n            controlnetapply_172 = controlnetapply.apply_controlnet(\\n                strength=0.5,\\n                conditioning=get_value_at_index(cliptextencode_169, 0),\\n                control_net=get_value_at_index(controlnetloader_174, 0),\\n                image=get_value_at_index(cannyedgepreprocessor_177, 0),\\n            )\\n\\n            aio_preprocessor_176 = aio_preprocessor.execute(\\n                preprocessor=\\\"DepthAnythingV2Preprocessor\\\",\\n                resolution=512,\\n                image=get_value_at_index(image_batch_230, 0),\\n            )\\n\\n            controlnetapply_173 = controlnetapply.apply_controlnet(\\n                strength=0.55,\\n                conditioning=get_value_at_index(controlnetapply_172, 0),\\n                control_net=get_value_at_index(controlnetloader_175, 0),\\n                image=get_value_at_index(aio_preprocessor_176, 0),\\n            )\\n\\n            emptylatentimage_167 = emptylatentimage.generate(\\n                width=512,\\n                height=512,\\n                batch_size=4,\\n            )\\n            torch.cuda.empty_cache()\\n            try:\\n                ksampler_1 = ksampler.sample(\\n                    seed=random.randint(1, 2**64),\\n                    steps=25,\\n                    cfg=7,\\n                    sampler_name=\\\"euler\\\",\\n                    scheduler=\\\"karras\\\",\\n                    denoise=1,\\n                    model=get_value_at_index(ipadapteradvanced_37, 0),\\n                    positive=get_value_at_index(controlnetapply_173, 0),\\n                    negative=get_value_at_index(cliptextencode_170, 0),\\n                    latent_image=get_value_at_index(emptylatentimage_167, 0),\\n                )\\n\\n                vaedecode_1 = vaedecode.decode(\\n                    samples=get_value_at_index(ksampler_1, 0),\\n                    vae=get_value_at_index(checkpointloadersimple_166, 2),\\n                )\\n            except Exception as e:\\n                logger.error(f\\\"ksampler or vaedecode Error: {e}\\\")\\n                raise Exception(f\\\"ksampler or vaedecode Error: {e}\\\")\\n\\n            if style == \\\"Line Art\\\":\\n                imageluminancedetector_1 = imageluminancedetector.execute(\\n                    gamma_correction=1,\\n                    resolution=512,\\n                    image=get_value_at_index(vaedecode_1, 0),\\n                )\\n                wait_to_save = imageluminancedetector_1\\n            else:\\n                wait_to_save = vaedecode_1\\n\\n            time_suffix = datetime.now().strftime(\\\"%Y%m%d_%H%M%S\\\")\\n            saveimage_1 = saveimage.save_images(\\n                filename_prefix=f\\\"style_vision_{time_suffix}\\\",\\n                images=get_value_at_index(wait_to_save, 0),\\n            )\\n            style_image_urls = []\\n            for i in range(4):\\n                filename = saveimage_1['ui']['images'][i]['filename']\\n                temp_image_url = obs_client.obs_upload_file(os.path.join(OUTPUT_PATH, filename), dir=f\\\"{CURRENT_MODEL}/{task_id}\\\")\\n                shutil.move(os.path.join(OUTPUT_PATH, filename), os.path.join(SAVE_PATH, CURRENT_MODEL, task_id, filename))\\n                style_image_urls.append(temp_image_url)\\n\\n            json_file = os.path.join(SAVE_PATH, CURRENT_MODEL, task_id, f\\\"images_style_{time_suffix}.json\\\")\\n            with open(json_file, \\\"w\\\") as f:\\n                image_data = {\\n                    \\\"style\\\": style +\\\":\\\"+style_description,\\n                    \\\"style_negative\\\": \\\"negative:\\\"+negative_description,\\n                    \\\"style_image_list\\\": style_image_list[style],\\n                    \\\"image_with_style\\\": image_urls,\\n                    \\\"style_image_urls\\\": style_image_urls\\n                }\\n                json.dump(image_data, f, indent=4, ensure_ascii=False) \\n            image_info = obs_client.obs_upload_file(file_path=json_file, dir=f\\\"{CURRENT_MODEL}/{task_id}\\\")\\n            print(f\\\"Saved image info to {image_info}\\\")\\n\\n            torch.cuda.empty_cache()\\n            return ImageResponse(code=200, message=\\\"success\\\", data=style_image_urls)\\n        \\n        except Exception as e:\\n            logger.error(f\\\"Style Transfer Error: {e}\\\")\\n            return ImageResponse(code=500, message=f\\\"error:{e}\\\", data=[])\\n\\n\\nif __name__ == \\\"__main__\\\":\\n    style_transfer()\\n\\n\\nimport sys, os\\nfrom pydantic import BaseModel\\nfrom typing import Any, Mapping, Sequence, Union, List\\n\\nimport inspect\\nimport os, sys\\nimport logging\\nfrom argostranslate import translate\\nfrom openai import OpenAI, BadRequestError\\nfrom fastapi.openapi.docs import get_swagger_ui_html\\n\\nINPUT_PATH = \\\"/root/ComfyUI/input\\\"\\nOUTPUT_PATH = \\\"/root/ComfyUI/output\\\"\\nSAVE_PATH = \\\"/root/autodl-tmp/data_store\\\"\\nCURRENT_MODEL = \\\"story_diffusion\\\"\\nos.makedirs(SAVE_PATH, exist_ok=True)\\n\\n\\n# 配置日志记录\\nlogging.basicConfig(level=logging.INFO)\\nlogger = logging.getLogger(__name__)\\n# 添加文件处理器\\nlog_file_path = \\\"/root/ComfyUI_Story/MYROUTER/logfile.log\\\"\\nfile_handler = logging.FileHandler(log_file_path)\\nfile_handler.setLevel(logging.INFO)\\n# 创建日志格式器并将其添加到文件处理器\\nformatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')\\nfile_handler.setFormatter(formatter)\\n# 将文件处理器添加到日志记录器\\nlogger.addHandler(file_handler)\\n\\nclass ImageResponse(BaseModel):\\n    code: int = 0\\n    data: Union[str, List[str]] = []\\n    message: str = \\\"\\\"\\n\\n\\nimage_style = {\\n    \\\"Anime Style\\\": \\\"Anime style, key visual, vibrant, studio anime, highly detailed, intricate character design, dynamic poses, vivid color palette, expressive facial expressions, cinematic lighting, sharp line art, atmospheric background, stylized effects, fluid motion, intense emotions, high contrast shading, elaborate costume details, fantasy elements, glowing accents, epic scene composition\\\",\\n    \\\"Photographic\\\": \\\"35mm photograph,film,bokeh,professional,4k,highly detailed\\\",\\n    \\\"Digital Art\\\": \\\"Digital artwork,illustrative,painterly,matte painting,highly detailed\\\",\\n    \\\"Comic Book\\\": \\\"Graphic illustration, comic art, graphic novel art, vibrant, highly detailed, bold line work, dynamic panel composition, exaggerated perspectives, expressive character designs, vivid color schemes, dramatic shading, textured backgrounds, action-packed scenes, stylized typography, narrative storytelling elements, contrasting shadows, impactful visual effects, onomatopoeic sound words, superhero themes, intricate costume designs, classic comic book aesthetics\\\",\\n    \\\"Fantasy Art\\\": \\\"magnificent,celestial,ethereal,painterly,epic,majestic,magical,fantasy art,cover art,dreamy\\\",\\n    \\\"Analog Film\\\": \\\"faded film,desaturated,35mm photo,grainy,vignette,vintage,Kodachrome,Lomography,stained,highly detailed,found footage\\\",\\n    \\\"Neon Punk\\\": \\\"cyberpunk,vaporwave,neon,vibes,vibrant,stunningly beautiful,crisp,detailed,sleek,ultramodern,magenta highlights,dark purple shadows,high contrast,cinematic,ultra detailed,intricate,professional\\\",\\n    \\\"Isometric\\\": \\\"vibrant, beautiful, crisp, detailed, ultra detailed, intricate\\\",\\n    \\\"Low Poly\\\": \\\"low-poly game art, polygon mesh, jagged, blocky, wireframe edges,centered composition\\\",\\n    \\\"Line Art\\\": \\\"professional, sleek, modern, minimalist, graphic, line art,vector graphics\\\",\\n    \\\"Cinematic\\\": \\\"shallow depth of field, vignette, highly detailed, high budget Hollywood movie, bokeh, cinemascope, moody, epic, gorgeous, film grain, grainy\\\",\\n    \\\"3D Model\\\": \\\"Octane render, highly detailed, volumetric, dramatic lighting, hyper-realistic textures, photorealistic materials, intricate geometry, complex surface detailing, cinematic depth of field, dynamic shadows, subsurface scattering, physically accurate reflections, precise caustics, atmospheric fog, soft gradients, high dynamic range, seamless integration, immersive environment, high-resolution rendering, polished finish, professional studio quality\\\",\\n    \\\"Graffiti Style\\\": \\\"Street art, vibrant, urban, detailed, tag, mural, bold color contrasts, dynamic spray paint textures, intricate line work, expressive lettering, graffiti characters, distressed surfaces, raw urban aesthetic, layered paint strokes, mixed media elements, grunge textures, abstract shapes, exaggerated proportions, vibrant neon highlights, chaotic composition, street culture vibe, wall textures, concrete backgrounds, rebellious theme, hip-hop influence\\\",\\n    \\\"Advertising Poster\\\": \\\"Professional, modern, product-focused, commercial,eye-catching, highly detailed\\\",\\n    \\\"Craft Clay\\\": \\\"Sculpture, clay art, centered composition, claymation, handcrafted texture, detailed sculpting, organic shapes, soft earthy tones, subtle imperfections, realistic clay surface, artisan craftsmanship, smooth curves, vibrant glaze accents, tactile feel, fine detailing, whimsical character designs, playful expressions, layered modeling, animated poses, lifelike contours, warm lighting, studio setting, cozy atmosphere, artisanal quality\\\",\\n    \\\"Pixel Art\\\": \\\"low-res, blocky, pixel art style,8-bit graphics, pixel art 4k wallpaper. Incredible pixel art details. Pixel art. Steam waves. Detailed unreal engine pixel art.\\\",\\n    \\\"Oil Painting\\\": \\\"Create a vibrant impressionist painting that captures the essence of a sunlit garden in spring. Use loose brushstrokes to convey the play of light and shadow on colorful flowers and lush greenery. Incorporate soft, swirling colors to evoke a sense of movement and tranquility, reminiscent of Monet's style.\\\",\\n}\\n\\nstyle_image_list = {\\n    \\\"Anime Style\\\" : [\\\"anime0.png\\\", \\\"anime1.png\\\", \\\"anime2.png\\\", \\\"anime3.png\\\"],\\n    \\\"Comic Book\\\" : [\\\"comic0.png\\\", \\\"comic1.png\\\", \\\"comic2.png\\\", \\\"comic3.png\\\"],\\n    \\\"Line Art\\\": [\\\"lineart0.png\\\", \\\"lineart1.png\\\", \\\"lineart2.png\\\", \\\"lineart3.png\\\"],\\n    \\\"Craft Clay\\\": [\\\"clay0.png\\\", \\\"clay1.png\\\", \\\"clay2.png\\\", \\\"clay3.png\\\"],\\n    \\\"Neon Punk\\\": [\\\"neon0.png\\\", \\\"neon1.png\\\", \\\"neon2.png\\\", \\\"neon3.png\\\"],\\n    \\\"Graffiti Style\\\": [\\\"graffiti0.png\\\", \\\"graffiti1.png\\\", \\\"graffiti2.png\\\", \\\"graffiti3.png\\\"],\\n    \\\"3D Model\\\": [\\\"threed0.png\\\", \\\"threed1.png\\\", \\\"threed2.png\\\", \\\"threed3.png\\\"],\\n    \\\"Oil Painting\\\": [\\\"oil0.png\\\", \\\"oil1.png\\\", \\\"oil2.png\\\", \\\"oil3.png\\\"],\\n    \\\"Pixel Art\\\": [\\\"pixart0.png\\\", \\\"pixart1.png\\\", \\\"pixart2.png\\\", \\\"pixart3.png\\\"],\\n}\\n\\ndef trans_zh2en(prompt):\\n    translation = translate.get_translation_from_codes(\\\"zh\\\", \\\"en\\\")\\n    return translation.translate(prompt)\\n\\nclient = OpenAI(\\n    api_key=\\\"5f8d4b0d8aae79363f13773256f216aa.hAUYfEWDW2qg7G8o\\\",\\n    base_url=\\\"https://open.bigmodel.cn/api/paas/v4/\\\"\\n)\\n\\ndef text2text(prompt):\\n    completion = client.chat.completions.create(\\n        model=\\\"glm-4\\\",\\n        messages=[\\n            {\\\"role\\\": \\\"user\\\", \\\"content\\\": prompt}\\n        ],\\n        top_p=0.7,\\n        temperature=0.9\\n    )\\n\\n    return completion.choices[0].message.content\\n\\n# FastAPI的一些函数\\ndef get_function_default_args(func):\\n    '''获取函数默认参数'''\\n    sign = inspect.signature(func)\\n    return {\\n        k: v.default\\n        for k, v in sign.parameters.items()\\n        if v.default is not inspect.Parameter.empty\\n    }\\n\\ndef swagger_monkey_patch(*args, **kwargs):\\n    \\\"\\\"\\\"\\n    Wrap the function which is generating the HTML for the /docs endpoint and\\n    overwrite the default values for the swagger js and css.\\n    \\\"\\\"\\\"\\n    param_dict = get_function_default_args(get_swagger_ui_html)\\n    swagger_js_url = param_dict['swagger_js_url'].replace('https://cdn.jsdelivr.net/npm/', 'https://unpkg.com/')\\n    swagger_css_url = param_dict['swagger_css_url'].replace('https://cdn.jsdelivr.net/npm/', 'https://unpkg.com/')\\n    return get_swagger_ui_html(\\n        *args, **kwargs,\\n        swagger_js_url=swagger_js_url,\\n        swagger_css_url=swagger_css_url\\n    )\\n\\n\\n# Comfyui的组件和节点函数\\ndef get_value_at_index(obj: Union[Sequence, Mapping], index: int) -> Any:\\n    \\\"\\\"\\\"Returns the value at the given index of a sequence or mapping.\\n\\n    If the object is a sequence (like list or string), returns the value at the given index.\\n    If the object is a mapping (like a dictionary), returns the value at the index-th key.\\n\\n    Some return a dictionary, in these cases, we look for the \\\"results\\\" key\\n\\n    Args:\\n        obj (Union[Sequence, Mapping]): The object to retrieve the value from.\\n        index (int): The index of the value to retrieve.\\n\\n    Returns:\\n        Any: The value at the given index.\\n\\n    Raises:\\n        IndexError: If the index is out of bounds for the object and the object is not a mapping.\\n    \\\"\\\"\\\"\\n    try:\\n        return obj[index]\\n    except KeyError:\\n        return obj[\\\"result\\\"][index]\\n\\ndef find_path(name: str, path: str = None) -> str:\\n    \\\"\\\"\\\"\\n    Recursively looks at parent folders starting from the given path until it finds the given name.\\n    Returns the path as a Path object if found, or None otherwise.\\n    \\\"\\\"\\\"\\n    # If no path is given, use the current working directory\\n    if path is None:\\n        path = os.getcwd()\\n\\n    # Check if the current directory contains the name\\n    if name in os.listdir(path):\\n        path_name = os.path.join(path, name)\\n        print(f\\\"{name} found: {path_name}\\\")\\n        return path_name\\n\\n    # Get the parent directory\\n    parent_directory = os.path.dirname(path)\\n\\n    # If the parent directory is the same as the current directory, we've reached the root and stop the search\\n    if parent_directory == path:\\n        return None\\n\\n    # Recursively call the function with the parent directory\\n    return find_path(name, parent_directory)\\n\\n\\ndef add_comfyui_directory_to_sys_path() -> None:\\n    \\\"\\\"\\\"\\n    Add 'ComfyUI' to the sys.path\\n    \\\"\\\"\\\"\\n    comfyui_path = find_path(\\\"ComfyUI\\\")\\n    if comfyui_path is not None and os.path.isdir(comfyui_path):\\n        sys.path.append(comfyui_path)\\n        print(f\\\"'{comfyui_path}' added to sys.path\\\")\\n\\ndef import_custom_nodes() -> None:\\n    \\\"\\\"\\\"Find all custom nodes in the custom_nodes folder and add those node objects to NODE_CLASS_MAPPINGS\\n\\n    This function sets up a new asyncio event loop, initializes the PromptServer,\\n    creates a PromptQueue, and initializes the custom nodes.\\n    \\\"\\\"\\\"\\n    import asyncio\\n    import execution\\n    from nodes import init_extra_nodes\\n    import server\\n\\n    # Creating a new event loop and setting it as the default loop\\n    loop = asyncio.new_event_loop()\\n    asyncio.set_event_loop(loop)\\n\\n    # Creating an instance of PromptServer with the loop\\n    server_instance = server.PromptServer(loop)\\n    execution.PromptQueue(server_instance)\\n\\n    # Initializing custom nodes\\n    init_extra_nodes()\\n\\nimport torch\\nfrom diffusers import StableDiffusionXLInpaintPipeline\\nfrom PIL import Image\\n\\nfrom ip_adapter import IPAdapterXL\\n\\nbase_model_path = \\\"diffusers/stable-diffusion-xl-1.0-inpainting-0.1\\\"\\nimage_encoder_path = \\\"sdxl_models/image_encoder\\\"\\nip_ckpt = \\\"sdxl_models/ip-adapter_sdxl.bin\\\"\\ndevice = \\\"cuda\\\"\\n\\n# load SDXL pipeline\\npipe = StableDiffusionXLInpaintPipeline.from_pretrained(\\n    base_model_path,\\n    torch_dtype=torch.float16,\\n    variant=\\\"fp16\\\",\\n    use_safetensors=True,\\n)\\npipe.enable_vae_tiling()\\n\\n# load ip-adapter\\n# target_blocks=[\\\"block\\\"] for original IP-Adapter\\n# target_blocks=[\\\"up_blocks.0.attentions.1\\\"] for style blocks only\\n# target_blocks = [\\\"up_blocks.0.attentions.1\\\", \\\"down_blocks.2.attentions.1\\\"] # for style+layout blocks\\nip_model = IPAdapterXL(pipe, image_encoder_path, ip_ckpt, device, target_blocks=[\\\"up_blocks.0.attentions.1\\\"])\\n\\nimage = \\\"./assets/5.jpg\\\"\\nimage = Image.open(image)\\nimage.resize((512, 512))\\n\\ninit_image = Image.open(\\\"./assets/overture-creations-5sI6fQgYIuo.png\\\").convert(\\\"RGB\\\")\\nmask_image = Image.open(\\\"./assets/overture-creations-5sI6fQgYIuo_mask_inverse.png\\\").convert(\\\"RGB\\\")\\n\\n# generate image\\nimages = ip_model.generate(pil_image=image,\\n                            prompt=\\\"a dog sitting on, masterpiece, best quality, high quality\\\",\\n                            negative_prompt= \\\"text, watermark, lowres, low quality, worst quality, deformed, glitch, low contrast, noisy, saturation, blurry\\\",\\n                            scale=2.0,\\n                            guidance_scale=8,\\n                            num_samples=1,\\n                            num_inference_steps=30, \\n                            image=init_image,\\n                            mask_image=mask_image,\\n                            strength=0.99\\n                            )\\n\\nimages[0].save(\\\"result.png\\\")\\n\\nimport torch\\nfrom diffusers import StableDiffusionPipeline, UniPCMultistepScheduler\\nfrom PIL import Image\\n\\nfrom ip_adapter import IPAdapter\\n\\nbase_model_path = \\\"runwayml/stable-diffusion-v1-5\\\"\\nimage_encoder_path = \\\"models/image_encoder\\\"\\nip_ckpt = \\\"models/ip-adapter_sd15.bin\\\"\\ndevice = \\\"cuda\\\"\\n\\n# load SDXL pipeline\\npipe = StableDiffusionPipeline.from_pretrained(\\n    base_model_path,\\n    torch_dtype=torch.float16,\\n)\\npipe.scheduler = UniPCMultistepScheduler.from_config(pipe.scheduler.config)\\npipe.enable_vae_tiling()\\n\\n# load ip-adapter\\n# target_blocks=[\\\"block\\\"] for original IP-Adapter\\n# target_blocks=[\\\"up_blocks.1\\\"] for style blocks only (experimental, not obvious as SDXL)\\n# target_blocks = [\\\"down_blocks.2\\\", \\\"mid_block\\\", \\\"up_blocks.1\\\"] # for style+layout blocks (experimental, not obvious as SDXL)\\nip_model = IPAdapter(pipe, image_encoder_path, ip_ckpt, device, target_blocks=[\\\"block\\\"])\\n\\nimage = \\\"./assets/3.jpg\\\"\\nimage = Image.open(image)\\nimage.resize((512, 512))\\n\\n# set negative content\\nneg_content = \\\"a girl\\\"\\nneg_content_scale = 0.8\\nif neg_content is not None:\\n    from transformers import CLIPTextModelWithProjection, CLIPTokenizer\\n    text_encoder = CLIPTextModelWithProjection.from_pretrained(\\\"laion/CLIP-ViT-H-14-laion2B-s32B-b79K\\\").to(pipe.device, \\n                                                                                                           dtype=pipe.dtype)\\n    tokenizer = CLIPTokenizer.from_pretrained(\\\"laion/CLIP-ViT-H-14-laion2B-s32B-b79K\\\")\\n\\n    tokens = tokenizer([neg_content], return_tensors='pt').to(pipe.device)\\n    neg_content_emb = text_encoder(**tokens).text_embeds\\n    neg_content_emb *= neg_content_scale\\nelse:\\n    neg_content_emb = None\\n\\n# generate image with content subtraction\\nimages = ip_model.generate(pil_image=image,\\n                           prompt=\\\"a cat, masterpiece, best quality, high quality\\\",\\n                           negative_prompt= \\\"text, watermark, lowres, low quality, worst quality, deformed, glitch, low contrast, noisy, saturation, blurry\\\",\\n                           scale=1.0,\\n                           guidance_scale=5,\\n                           num_samples=1,\\n                           num_inference_steps=30, \\n                           seed=42,\\n                           neg_content_emb=neg_content_emb,\\n                          )\\n\\nimages[0].save(\\\"result.png\\\")\\n\\n'down_blocks.1.attentions.0.transformer_blocks.0.attn1.processor',\\n'down_blocks.1.attentions.0.transformer_blocks.0.attn2.processor',\\n'down_blocks.1.attentions.0.transformer_blocks.1.attn1.processor',\\n'down_blocks.1.attentions.0.transformer_blocks.1.attn2.processor',\\n'down_blocks.1.attentions.1.transformer_blocks.0.attn1.processor',\\n'down_blocks.1.attentions.1.transformer_blocks.0.attn2.processor',\\n'down_blocks.1.attentions.1.transformer_blocks.1.attn1.processor', \\n'down_blocks.1.attentions.1.transformer_blocks.1.attn2.processor', \\n'down_blocks.2.attentions.0.transformer_blocks.0.attn1.processor', \\n'down_blocks.2.attentions.0.transformer_blocks.0.attn2.processor', \\n'down_blocks.2.attentions.0.transformer_blocks.1.attn1.processor', \\n'down_blocks.2.attentions.0.transformer_blocks.1.attn2.processor', \\n'down_blocks.2.attentions.0.transformer_blocks.2.attn1.processor', \\n'down_blocks.2.attentions.0.transformer_blocks.2.attn2.processor', \\n'down_blocks.2.attentions.0.transformer_blocks.3.attn1.processor', \\n'down_blocks.2.attentions.0.transformer_blocks.3.attn2.processor', \\n'down_blocks.2.attentions.0.transformer_blocks.4.attn1.processor', \\n'down_blocks.2.attentions.0.transformer_blocks.4.attn2.processor', \\n'down_blocks.2.attentions.0.transformer_blocks.5.attn1.processor', \\n'down_blocks.2.attentions.0.transformer_blocks.5.attn2.processor', \\n'down_blocks.2.attentions.0.transformer_blocks.6.attn1.processor', \\n'down_blocks.2.attentions.0.transformer_blocks.6.attn2.processor', \\n'down_blocks.2.attentions.0.transformer_blocks.7.attn1.processor', \\n'down_blocks.2.attentions.0.transformer_blocks.7.attn2.processor', \\n'down_blocks.2.attentions.0.transformer_blocks.8.attn1.processor', \\n'down_blocks.2.attentions.0.transformer_blocks.8.attn2.processor', \\n'down_blocks.2.attentions.0.transformer_blocks.9.attn1.processor',\\n'down_blocks.2.attentions.0.transformer_blocks.9.attn2.processor', \\n'down_blocks.2.attentions.1.transformer_blocks.0.attn1.processor',\\n'down_blocks.2.attentions.1.transformer_blocks.0.attn2.processor', \\n'down_blocks.2.attentions.1.transformer_blocks.1.attn1.processor', \\n'down_blocks.2.attentions.1.transformer_blocks.1.attn2.processor', \\n'down_blocks.2.attentions.1.transformer_blocks.2.attn1.processor', \\n'down_blocks.2.attentions.1.transformer_blocks.2.attn2.processor', \\n'down_blocks.2.attentions.1.transformer_blocks.3.attn1.processor', \\n'down_blocks.2.attentions.1.transformer_blocks.3.attn2.processor', \\n'down_blocks.2.attentions.1.transformer_blocks.4.attn1.processor', \\n'down_blocks.2.attentions.1.transformer_blocks.4.attn2.processor', \\n'down_blocks.2.attentions.1.transformer_blocks.5.attn1.processor', \\n'down_blocks.2.attentions.1.transformer_blocks.5.attn2.processor', \\n'down_blocks.2.attentions.1.transformer_blocks.6.attn1.processor', \\n'down_blocks.2.attentions.1.transformer_blocks.6.attn2.processor', \\n'down_blocks.2.attentions.1.transformer_blocks.7.attn1.processor', \\n'down_blocks.2.attentions.1.transformer_blocks.7.attn2.processor', \\n'down_blocks.2.attentions.1.transformer_blocks.8.attn1.processor', \\n'down_blocks.2.attentions.1.transformer_blocks.8.attn2.processor', \\n'down_blocks.2.attentions.1.transformer_blocks.9.attn1.processor', \\n'down_blocks.2.attentions.1.transformer_blocks.9.attn2.processor', \\n'up_blocks.0.attentions.0.transformer_blocks.0.attn1.processor', \\n'up_blocks.0.attentions.0.transformer_blocks.0.attn2.processor', \\n'up_blocks.0.attentions.0.transformer_blocks.1.attn1.processor', \\n'up_blocks.0.attentions.0.transformer_blocks.1.attn2.processor', \\n'up_blocks.0.attentions.0.transformer_blocks.2.attn1.processor', \\n'up_blocks.0.attentions.0.transformer_blocks.2.attn2.processor', \\n'up_blocks.0.attentions.0.transformer_blocks.3.attn1.processor', \\n'up_blocks.0.attentions.0.transformer_blocks.3.attn2.processor', \\n'up_blocks.0.attentions.0.transformer_blocks.4.attn1.processor', \\n'up_blocks.0.attentions.0.transformer_blocks.4.attn2.processor', \\n'up_blocks.0.attentions.0.transformer_blocks.5.attn1.processor', \\n'up_blocks.0.attentions.0.transformer_blocks.5.attn2.processor', \\n'up_blocks.0.attentions.0.transformer_blocks.6.attn1.processor', \\n'up_blocks.0.attentions.0.transformer_blocks.6.attn2.processor', \\n'up_blocks.0.attentions.0.transformer_blocks.7.attn1.processor', \\n'up_blocks.0.attentions.0.transformer_blocks.7.attn2.processor', \\n'up_blocks.0.attentions.0.transformer_blocks.8.attn1.processor', \\n'up_blocks.0.attentions.0.transformer_blocks.8.attn2.processor', \\n'up_blocks.0.attentions.0.transformer_blocks.9.attn1.processor', \\n'up_blocks.0.attentions.0.transformer_blocks.9.attn2.processor', \\n'up_blocks.0.attentions.1.transformer_blocks.0.attn1.processor', \\n'up_blocks.0.attentions.1.transformer_blocks.0.attn2.processor', \\n'up_blocks.0.attentions.1.transformer_blocks.1.attn1.processor', \\n'up_blocks.0.attentions.1.transformer_blocks.1.attn2.processor', \\n'up_blocks.0.attentions.1.transformer_blocks.2.attn1.processor', \\n'up_blocks.0.attentions.1.transformer_blocks.2.attn2.processor', \\n'up_blocks.0.attentions.1.transformer_blocks.3.attn1.processor', \\n'up_blocks.0.attentions.1.transformer_blocks.3.attn2.processor', \\n'up_blocks.0.attentions.1.transformer_blocks.4.attn1.processor', \\n'up_blocks.0.attentions.1.transformer_blocks.4.attn2.processor', \\n'up_blocks.0.attentions.1.transformer_blocks.5.attn1.processor', \\n'up_blocks.0.attentions.1.transformer_blocks.5.attn2.processor', \\n'up_blocks.0.attentions.1.transformer_blocks.6.attn1.processor', \\n'up_blocks.0.attentions.1.transformer_blocks.6.attn2.processor', \\n'up_blocks.0.attentions.1.transformer_blocks.7.attn1.processor', \\n'up_blocks.0.attentions.1.transformer_blocks.7.attn2.processor', \\n'up_blocks.0.attentions.1.transformer_blocks.8.attn1.processor', \\n'up_blocks.0.attentions.1.transformer_blocks.8.attn2.processor', \\n'up_blocks.0.attentions.1.transformer_blocks.9.attn1.processor', \\n'up_blocks.0.attentions.1.transformer_blocks.9.attn2.processor', \\n'up_blocks.0.attentions.2.transformer_blocks.0.attn1.processor', \\n'up_blocks.0.attentions.2.transformer_blocks.0.attn2.processor', \\n'up_blocks.0.attentions.2.transformer_blocks.1.attn1.processor', \\n'up_blocks.0.attentions.2.transformer_blocks.1.attn2.processor', \\n'up_blocks.0.attentions.2.transformer_blocks.2.attn1.processor', \\n'up_blocks.0.attentions.2.transformer_blocks.2.attn2.processor', \\n'up_blocks.0.attentions.2.transformer_blocks.3.attn1.processor', \\n'up_blocks.0.attentions.2.transformer_blocks.3.attn2.processor', \\n'up_blocks.0.attentions.2.transformer_blocks.4.attn1.processor', \\n'up_blocks.0.attentions.2.transformer_blocks.4.attn2.processor', \\n'up_blocks.0.attentions.2.transformer_blocks.5.attn1.processor', \\n'up_blocks.0.attentions.2.transformer_blocks.5.attn2.processor', \\n'up_blocks.0.attentions.2.transformer_blocks.6.attn1.processor', \\n'up_blocks.0.attentions.2.transformer_blocks.6.attn2.processor', \\n'up_blocks.0.attentions.2.transformer_blocks.7.attn1.processor', \\n'up_blocks.0.attentions.2.transformer_blocks.7.attn2.processor', \\n'up_blocks.0.attentions.2.transformer_blocks.8.attn1.processor', \\n'up_blocks.0.attentions.2.transformer_blocks.8.attn2.processor', \\n'up_blocks.0.attentions.2.transformer_blocks.9.attn1.processor', \\n'up_blocks.0.attentions.2.transformer_blocks.9.attn2.processor', \\n'up_blocks.1.attentions.0.transformer_blocks.0.attn1.processor', \\n'up_blocks.1.attentions.0.transformer_blocks.0.attn2.processor', \\n'up_blocks.1.attentions.0.transformer_blocks.1.attn1.processor', \\n'up_blocks.1.attentions.0.transformer_blocks.1.attn2.processor', \\n'up_blocks.1.attentions.1.transformer_blocks.0.attn1.processor', \\n'up_blocks.1.attentions.1.transformer_blocks.0.attn2.processor', \\n'up_blocks.1.attentions.1.transformer_blocks.1.attn1.processor', \\n'up_blocks.1.attentions.1.transformer_blocks.1.attn2.processor', \\n'up_blocks.1.attentions.2.transformer_blocks.0.attn1.processor', \\n'up_blocks.1.attentions.2.transformer_blocks.0.attn2.processor', \\n'up_blocks.1.attentions.2.transformer_blocks.1.attn1.processor', \\n'up_blocks.1.attentions.2.transformer_blocks.1.attn2.processor', \\n'mid_block.attentions.0.transformer_blocks.0.attn1.processor', \\n'mid_block.attentions.0.transformer_blocks.0.attn2.processor', \\n'mid_block.attentions.0.transformer_blocks.1.attn1.processor', \\n'mid_block.attentions.0.transformer_blocks.1.attn2.processor', \\n'mid_block.attentions.0.transformer_blocks.2.attn1.processor', \\n'mid_block.attentions.0.transformer_blocks.2.attn2.processor', \\n'mid_block.attentions.0.transformer_blocks.3.attn1.processor', \\n'mid_block.attentions.0.transformer_blocks.3.attn2.processor', \\n'mid_block.attentions.0.transformer_blocks.4.attn1.processor', \\n'mid_block.attentions.0.transformer_blocks.4.attn2.processor', \\n'mid_block.attentions.0.transformer_blocks.5.attn1.processor', \\n'mid_block.attentions.0.transformer_blocks.5.attn2.processor', \\n'mid_block.attentions.0.transformer_blocks.6.attn1.processor', \\n'mid_block.attentions.0.transformer_blocks.6.attn2.processor', \\n'mid_block.attentions.0.transformer_blocks.7.attn1.processor', \\n'mid_block.attentions.0.transformer_blocks.7.attn2.processor', \\n'mid_block.attentions.0.transformer_blocks.8.attn1.processor', \\n'mid_block.attentions.0.transformer_blocks.8.attn2.processor', \\n'mid_block.attentions.0.transformer_blocks.9.attn1.processor', \\n'mid_block.attentions.0.transformer_blocks.9.attn2.processor'\\n\\nimport torch\\nfrom diffusers import ControlNetModel, StableDiffusionXLControlNetPipeline\\n\\nimport cv2\\nimport os\\nos.environ[\\\"HF_ENDPOINT\\\"] = \\\"https://hf-mirror.com\\\"\\nfrom PIL import Image\\n\\nfrom .ip_adapter import IPAdapterXL\\n\\nbase_model_path = \\\"/root/autodl-tmp/InstantStyle_Model/stabilityai/stable-diffusion-xl-base-1.0\\\" # \\\"stabilityai/stable-diffusion-xl-base-1.0\\\"\\nimage_encoder_path =\\\"/root/autodl-tmp/InstantStyle_Model/sdxl_models/image_encoder\\\" # \\\"sdxl_models/image_encoder\\\"\\nip_ckpt = \\\"/root/autodl-tmp/InstantStyle_Model/sdxl_models/ip-adapter_sdxl.bin\\\" #\\\"sdxl_models/ip-adapter_sdxl.bin\\\"\\ndevice = \\\"cuda\\\"\\n\\ncontrolnet_path =\\\"/root/autodl-tmp/InstantStyle_Model/diffusers/controlnet-canny-sdxl-1.0\\\" # \\\"diffusers/controlnet-canny-sdxl-1.0\\\"\\ncontrolnet = ControlNetModel.from_pretrained(controlnet_path, use_safetensors=False, torch_dtype=torch.float16).to(device)\\n\\n# load SDXL pipeline\\npipe = StableDiffusionXLControlNetPipeline.from_pretrained(\\n    base_model_path,\\n    controlnet=controlnet,\\n    torch_dtype=torch.float16,\\n    add_watermarker=False,\\n)\\npipe.enable_xformers_memory_efficient_attention()\\n\\n\\n# load ip-adapter\\ntarget_blocks=[\\\"block\\\"] # for original IP-Adapter\\n# target_blocks=[\\\"up_blocks.0.attentions.1\\\"] # for style blocks only\\n# target_blocks = [\\\"up_blocks.0.attentions.1\\\", \\\"down_blocks.2.attentions.1\\\"] # for style+layout blocks\\nip_model = IPAdapterXL(pipe, image_encoder_path, ip_ckpt, device, target_blocks=target_blocks)\\n\\n\\ndef style_transfer(image_paths:list[str], style_image_path:str):\\n    # style image\\n    if style_image_path is None:\\n        style_image = \\\"/root/ComfyUI/MYROUTER/routers/InstantStyle/assets/style/R.jpg\\\"\\n    else:\\n        style_image = style_image_path\\n    style_image = Image.open(style_image)\\n    style_image.resize((720, 720))\\n\\n    # 加载和处理图像\\n    target_images = image_paths # [\\\"./assets/test/ComfyUI_00492_.png\\\",\\\"./assets/test/ComfyUI_00493_.png\\\",\\\"./assets/test/ComfyUI_00494_.png\\\", \\\"./assets/test/ComfyUI_00495_.png\\\"]\\n    canny_maps = [Image.fromarray(cv2.cvtColor(cv2.Canny(cv2.imread(img), 30, 200), cv2.COLOR_BGR2RGB)) for img in target_images]\\n    # style_images = [\\\"/root/InstantStyle/assets/style/H.jpg\\\", \\\"/root/InstantStyle/assets/style/R.jpg\\\", \\\"/root/InstantStyle/assets/style/R.jpg\\\", \\\"/root/InstantStyle/assets/style/L.jpg\\\"]\\n    # style_image = [Image.open(img) for img in style_images]\\n    # generate image\\n    images = ip_model.generate(pil_image=style_image,\\n                            prompt=None,\\n                            negative_prompt= \\\"text, watermark, lowres, low quality, worst quality, deformed, glitch, low contrast, noisy, saturation, blurry\\\",\\n                            scale=1.0,\\n                            guidance_scale=5,\\n                            num_samples=4,\\n                            num_inference_steps=35, \\n                            seed=42,\\n                            image=canny_maps,\\n                            controlnet_conditioning_scale=0.6,\\n                            )\\n\\n    images[0].save(\\\"result.png\\\")\\n    images[1].save(\\\"result1.png\\\")\\n    images[2].save(\\\"result2.png\\\")\\n    images[3].save(\\\"result3.png\\\")\\n\\nif __name__ == \\\"__main__\\\":\\n    pass\\n\\nimport torch\\nfrom diffusers import StableDiffusionXLPipeline\\nfrom PIL import Image\\n\\nfrom ip_adapter import IPAdapterPlusXL\\n\\nbase_model_path = \\\"stabilityai/stable-diffusion-xl-base-1.0\\\"\\nimage_encoder_path = \\\"models/image_encoder\\\"\\nip_ckpt = \\\"sdxl_models/ip-adapter-plus_sdxl_vit-h.bin\\\"\\ndevice = \\\"cuda\\\"\\n\\n# load SDXL pipeline\\npipe = StableDiffusionXLPipeline.from_pretrained(\\n    base_model_path,\\n    torch_dtype=torch.float16,\\n    add_watermarker=False,\\n)\\npipe.enable_vae_tiling()\\n\\n# load ip-adapter\\n# target_blocks=[\\\"block\\\"] for original IP-Adapter\\n# target_blocks=[\\\"up_blocks.0.attentions.1\\\"] for style blocks only\\n# target_blocks = [\\\"up_blocks.0.attentions.1\\\", \\\"down_blocks.2.attentions.1\\\"] # for style+layout blocks\\nip_model = IPAdapterPlusXL(pipe, image_encoder_path, ip_ckpt, device, num_tokens=16, target_blocks=[\\\"up_blocks.0.attentions.1\\\"])\\n\\nimage = \\\"./assets/0.jpg\\\"\\nimage = Image.open(image)\\nimage.resize((512, 512))\\n\\n# generate image\\nimages = ip_model.generate(pil_image=image,\\n                           prompt=\\\"a cat, masterpiece, best quality, high quality\\\",\\n                           negative_prompt= \\\"text, watermark, lowres, low quality, worst quality, deformed, glitch, low contrast, noisy, saturation, blurry\\\",\\n                           scale=1.0,\\n                           guidance_scale=5,\\n                           num_samples=1,\\n                           num_inference_steps=30, \\n                           seed=42,\\n                          )\\n\\nimages[0].save(\\\"result.png\\\")\\n\\n<div align=\\\"center\\\">\\n<h1>InstantStyle: Free Lunch towards Style-Preserving in Text-to-Image Generation</h1>\\n\\n[**Haofan Wang**](https://haofanwang.github.io/)<sup>*</sup> · [**Matteo Spinelli**](https://github.com/cubiq) · [**Qixun Wang**](https://github.com/wangqixun) · [**Xu Bai**](https://huggingface.co/baymin0220) · [**Zekui Qin**](https://github.com/ZekuiQin) · [**Anthony Chen**](https://antonioo-c.github.io/)\\n\\nInstantX Team \\n\\n<sup>*</sup>corresponding authors\\n\\n<a href='https://instantstyle.github.io/'><img src='https://img.shields.io/badge/Project-Page-green'></a>\\n<a href='https://arxiv.org/abs/2404.02733'><img src='https://img.shields.io/badge/Technique-Report-red'></a>\\n[![Hugging Face](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Space-red)](https://huggingface.co/spaces/InstantX/InstantStyle)\\n[![ModelScope](https://img.shields.io/badge/ModelScope-Studios-blue)](https://modelscope.cn/studios/instantx/InstantStyle/summary)\\n[![GitHub](https://img.shields.io/github/stars/InstantStyle/InstantStyle?style=social)](https://github.com/InstantStyle/InstantStyle)\\n\\n</div>\\n\\nInstantStyle is a general framework that employs two straightforward yet potent techniques for achieving an effective disentanglement of style and content from reference images.\\n\\n<!-- <img src='assets/pipe.png'> -->\\n\\n<div align=\\\"center\\\">\\n<img src='assets/page0.png' width = 900 >\\n</div>\\n\\n\\n## Principle\\n\\nSeparating Content from Image. Benefit from the good characterization of CLIP global features, after subtracting the content text fea- tures from the image features, the style and content can be explicitly decoupled. Although simple, this strategy is quite effective in mitigating content leakage.\\n<p align=\\\"center\\\">\\n  <img src=\\\"assets/subtraction.png\\\">\\n</p>\\n\\nInjecting into Style Blocks Only. Empirically, each layer of a deep network captures different semantic information the key observation in our work is that there exists two specific attention layers handling style. Specifically, we find up blocks.0.attentions.1 and down blocks.2.attentions.1 capture style (color, material, atmosphere) and spatial layout (structure, composition) respectively.\\n<p align=\\\"center\\\">\\n  <img src=\\\"assets/tree.png\\\">\\n</p>\\n\\n## Release\\n- [2024/07/06] 🔥 We release [CSGO](https://github.com/instantX-research/CSGO) page for content-style composition. Code will be released soon.\\n- [2024/07/01] 🔥 We release [InstantStyle-Plus](https://instantstyle-plus.github.io/) report for content preserving.\\n- [2024/04/29] 🔥 We support InstantStyle natively in diffusers, usage can be found [here](https://github.com/InstantStyle/InstantStyle?tab=readme-ov-file#use-in-diffusers)\\n- [2024/04/24] 🔥 InstantStyle for fast generation, find demos at [InstantStyle-SDXL-Lightning](https://huggingface.co/spaces/radames/InstantStyle-SDXL-Lightning) and [InstantStyle-Hyper-SDXL](https://huggingface.co/spaces/radames/InstantStyle-Hyper-SDXL).\\n- [2024/04/24] 🔥 We support [HiDiffusion](https://github.com/megvii-research/HiDiffusion) for generating highres images, find more information [here](https://github.com/InstantStyle/InstantStyle/tree/main?tab=readme-ov-file#high-resolution-generation).\\n- [2024/04/23] 🔥 InstantStyle has been natively supported in diffusers, more information can be found [here](https://github.com/huggingface/diffusers/pull/7668).\\n- [2024/04/20] 🔥 InstantStyle is supported in [Mikubill/sd-webui-controlnet](https://github.com/Mikubill/sd-webui-controlnet/discussions/2770).\\n- [2024/04/11] 🔥 We add the experimental distributed inference feature. Check it [here](https://github.com/InstantStyle/InstantStyle?tab=readme-ov-file#distributed-inference).\\n- [2024/04/10] 🔥 We support an [online demo](https://modelscope.cn/studios/instantx/InstantStyle/summary) on ModelScope.\\n- [2024/04/09] 🔥 We support an [online demo](https://huggingface.co/spaces/InstantX/InstantStyle) on Huggingface.\\n- [2024/04/09] 🔥 We support SDXL-inpainting, more information can be found [here](https://github.com/InstantStyle/InstantStyle/blob/main/infer_style_inpainting.py).\\n- [2024/04/08] 🔥 InstantStyle is supported in [AnyV2V](https://tiger-ai-lab.github.io/AnyV2V/) for stylized video-to-video editing, demo can be found [here](https://twitter.com/vinesmsuic/status/1777170927500787782).\\n- [2024/04/07] 🔥 We support image-based stylization, more information can be found [here](https://github.com/InstantStyle/InstantStyle/blob/main/infer_style_controlnet.py).\\n- [2024/04/07] 🔥 We support an experimental version for SD1.5, more information can be found [here](https://github.com/InstantStyle/InstantStyle/blob/main/infer_style_sd15.py).\\n- [2024/04/03] 🔥 InstantStyle is supported in [ComfyUI_IPAdapter_plus](https://github.com/cubiq/ComfyUI_IPAdapter_plus) developed by our co-author.\\n- [2024/04/03] 🔥 We release the [technical report](https://arxiv.org/abs/2404.02733).\\n\\n## Demos\\n\\n### Stylized Synthesis\\n\\n<p align=\\\"center\\\">\\n  <img src=\\\"assets/example1.png\\\">\\n  <img src=\\\"assets/example2.png\\\">\\n</p>\\n\\n### Image-based Stylized Synthesis\\n\\n<p align=\\\"center\\\">\\n  <img src=\\\"assets/example3.png\\\">\\n</p>\\n\\n### Comparison with Previous Works\\n\\n<p align=\\\"center\\\">\\n  <img src=\\\"assets/comparison.png\\\">\\n</p>\\n\\n## Download\\nFollow [IP-Adapter](https://github.com/tencent-ailab/IP-Adapter?tab=readme-ov-file#download-models) to download pre-trained checkpoints from [here](https://huggingface.co/h94/IP-Adapter).\\n\\n```\\ngit clone https://github.com/InstantStyle/InstantStyle.git\\ncd InstantStyle\\n\\n# download the models\\ngit lfs install\\ngit clone https://huggingface.co/h94/IP-Adapter\\nmv IP-Adapter/models models\\nmv IP-Adapter/sdxl_models sdxl_models\\n```\\n\\n## Usage\\n\\nOur method is fully compatible with [IP-Adapter](https://github.com/tencent-ailab/IP-Adapter). For feature subtraction, it only works for global feature instead of patch features. For SD1.5, you can find a demo at [infer_style_sd15.py](https://github.com/InstantStyle/InstantStyle/blob/main/infer_style_sd15.py), but we find that SD1.5 has weaker perception and understanding of style information, thus this demo is experimental only. All block names can be found in [attn_blocks.py](https://github.com/InstantStyle/InstantStyle/blob/main/attn_blocks.py) and [attn_blocks_sd15.py](https://github.com/InstantStyle/InstantStyle/blob/main/attn_blocks_sd15.py) for SDXL and SD1.5 respectively.\\n\\n```python\\nimport torch\\nfrom diffusers import StableDiffusionXLPipeline\\nfrom PIL import Image\\n\\nfrom ip_adapter import IPAdapterXL\\n\\nbase_model_path = \\\"stabilityai/stable-diffusion-xl-base-1.0\\\"\\nimage_encoder_path = \\\"sdxl_models/image_encoder\\\"\\nip_ckpt = \\\"sdxl_models/ip-adapter_sdxl.bin\\\"\\ndevice = \\\"cuda\\\"\\n\\n# load SDXL pipeline\\npipe = StableDiffusionXLPipeline.from_pretrained(\\n    base_model_path,\\n    torch_dtype=torch.float16,\\n    add_watermarker=False,\\n)\\n\\n# reduce memory consumption\\npipe.enable_vae_tiling()\\n\\n# load ip-adapter\\n# target_blocks=[\\\"block\\\"] for original IP-Adapter\\n# target_blocks=[\\\"up_blocks.0.attentions.1\\\"] for style blocks only\\n# target_blocks = [\\\"up_blocks.0.attentions.1\\\", \\\"down_blocks.2.attentions.1\\\"] # for style+layout blocks\\nip_model = IPAdapterXL(pipe, image_encoder_path, ip_ckpt, device, target_blocks=[\\\"up_blocks.0.attentions.1\\\"])\\n\\nimage = \\\"./assets/0.jpg\\\"\\nimage = Image.open(image)\\nimage.resize((512, 512))\\n\\n# generate image variations with only image prompt\\nimages = ip_model.generate(pil_image=image,\\n                            prompt=\\\"a cat, masterpiece, best quality, high quality\\\",\\n                            negative_prompt= \\\"text, watermark, lowres, low quality, worst quality, deformed, glitch, low contrast, noisy, saturation, blurry\\\",\\n                            scale=1.0,\\n                            guidance_scale=5,\\n                            num_samples=1,\\n                            num_inference_steps=30, \\n                            seed=42,\\n                            #neg_content_prompt=\\\"a rabbit\\\",\\n                            #neg_content_scale=0.5,\\n                          )\\n\\nimages[0].save(\\\"result.png\\\")\\n```\\n\\n## Use in diffusers\\nInstantStyle has already been integrated into [diffusers](https://huggingface.co/docs/diffusers/main/en/using-diffusers/ip_adapter#style--layout-control) (please make sure that you have installed diffusers>=0.28.0.dev0), making the usage significantly simpler. You can now control the per-transformer behavior of each IP-Adapter with the set_ip_adapter_scale() method, using a configuration dictionary as shown below:\\n\\n```python\\nfrom diffusers import StableDiffusionXLPipeline\\nfrom PIL import Image\\nimport torch\\n\\n# load SDXL pipeline\\npipe = StableDiffusionXLPipeline.from_pretrained(\\n    \\\"stabilityai/stable-diffusion-xl-base-1.0\\\",\\n    torch_dtype=torch.float16,\\n    add_watermarker=False,\\n)\\n\\n# load ip-adapter\\npipe.load_ip_adapter(\\\"h94/IP-Adapter\\\", subfolder=\\\"sdxl_models\\\", weight_name=\\\"ip-adapter_sdxl.bin\\\")\\npipe.enable_vae_tiling()\\n\\n# configure ip-adapter scales.\\nscale = {\\n    \\\"down\\\": {\\\"block_2\\\": [0.0, 1.0]},\\n    \\\"up\\\": {\\\"block_0\\\": [0.0, 1.0, 0.0]},\\n}\\npipeline.set_ip_adapter_scale(scale)\\n```\\n\\nIn this example. We set ```scale=1.0``` for IP-Adapter in the second transformer of down-part, block 2, and the second in up-part, block 0. Note that there are 2 transformers in down-part block 2 so the list is of length 2, and so do the up-part block 0. The rest IP-Adapter will have a zero scale which means disable them in all the other layers.\\n\\nWith the help of ```set_ip_adapter_scale()```, we can now configure IP-Adapters without a need of reloading them everytime we want to test the IP-Adapter behaviors.\\n\\n```python\\n# for original IP-Adapter\\nscale = 1.0\\npipeline.set_ip_adapter_scale(scale)\\n\\n# for style blocks only\\nscale = {\\n    \\\"up\\\": {\\\"block_0\\\": [0.0, 1.0, 0.0]},\\n}\\npipeline.set_ip_adapter_scale(scale)\\n```\\n\\n### Multiple IP-Adapter images with masks\\nYou can also load multiple IP-Adapters, together with multiple IP-Adapter images with masks for more precisely layout control just as that in [IP-Adapter](https://huggingface.co/docs/diffusers/main/en/using-diffusers/ip_adapter#ip-adapter-masking) do.\\n\\n```python\\nfrom diffusers import StableDiffusionXLPipeline\\nfrom diffusers.image_processor import IPAdapterMaskProcessor\\nfrom transformers import CLIPVisionModelWithProjection\\nfrom PIL import Image\\nimport torch\\n\\nimage_encoder = CLIPVisionModelWithProjection.from_pretrained(\\n    \\\"h94/IP-Adapter\\\", subfolder=\\\"models/image_encoder\\\", torch_dtype=torch.float16\\n).to(\\\"cuda\\\")\\n\\npipe = StableDiffusionXLPipeline.from_pretrained(\\n    \\\"RunDiffusion/Juggernaut-XL-v9\\\", torch_dtype=torch.float16, image_encoder=image_encoder, variant=\\\"fp16\\\"\\n).to(\\\"cuda\\\")\\n\\npipe.load_ip_adapter(\\n    [\\\"ostris/ip-composition-adapter\\\", \\\"h94/IP-Adapter\\\"],\\n    subfolder=[\\\"\\\", \\\"sdxl_models\\\"],\\n    weight_name=[\\n        \\\"ip_plus_composition_sdxl.safetensors\\\",\\n        \\\"ip-adapter_sdxl_vit-h.safetensors\\\",\\n    ],\\n    image_encoder_folder=None,\\n)\\n\\nscale_1 = {\\n    \\\"down\\\": [[0.0, 0.0, 1.0]],\\n    \\\"mid\\\": [[0.0, 0.0, 1.0]],\\n    \\\"up\\\": {\\\"block_0\\\": [[0.0, 0.0, 1.0], [1.0, 1.0, 1.0], [0.0, 0.0, 1.0]], \\\"block_1\\\": [[0.0, 0.0, 1.0]]},\\n}\\n# activate the first IP-Adapter in everywhere in the model,\\n# configure the second one for precise style control to each masked input.\\npipe.set_ip_adapter_scale([1.0, scale_1])\\n\\nprocessor = IPAdapterMaskProcessor()\\nfemale_mask = Image.open(\\\"./assets/female_mask.png\\\")\\nmale_mask = Image.open(\\\"./assets/male_mask.png\\\")\\nbackground_mask = Image.open(\\\"./assets/background_mask.png\\\")\\ncomposition_mask = Image.open(\\\"./assets/composition_mask.png\\\")\\nmask1 = processor.preprocess([composition_mask], height=1024, width=1024)\\nmask2 = processor.preprocess([female_mask, male_mask, background_mask], height=1024, width=1024)\\nmask2 = mask2.reshape(1, mask2.shape[0], mask2.shape[2], mask2.shape[3])   # output -> (1, 3, 1024, 1024)\\n\\nip_female_style = Image.open(\\\"./assets/ip_female_style.png\\\")\\nip_male_style = Image.open(\\\"./assets/ip_male_style.png\\\")\\nip_background = Image.open(\\\"./assets/ip_background.png\\\")\\nip_composition_image = Image.open(\\\"./assets/ip_composition_image.png\\\")\\n\\nimage = pipe(\\n    prompt=\\\"high quality, cinematic photo, cinemascope, 35mm, film grain, highly detailed\\\",\\n    negative_prompt=\\\"\\\",\\n    ip_adapter_image=[ip_composition_image, [ip_female_style, ip_male_style, ip_background]],\\n    cross_attention_kwargs={\\\"ip_adapter_masks\\\": [mask1, mask2]},\\n    guidance_scale=6.5,\\n    num_inference_steps=25,\\n).images[0]\\nimage\\n\\n```\\n\\n<p align=\\\"center\\\">\\n  <img src=\\\"assets/multi_instantstyle.png\\\">\\n</p>\\n\\n## High Resolution Generation\\nWe employ [HiDiffusion](https://github.com/megvii-research/HiDiffusion) to seamlessly generate high-resolution images, you can install via `pip install hidiffusion`.\\n\\n```python\\nfrom hidiffusion import apply_hidiffusion, remove_hidiffusion\\n\\n# reduce memory consumption\\npipe.enable_vae_tiling()\\n\\n# apply hidiffusion with a single line of code.\\napply_hidiffusion(pipe)\\n\\n...\\n\\n# generate image at higher resolution\\nimages = ip_model.generate(pil_image=image,\\n                           prompt=\\\"a cat, masterpiece, best quality, high quality\\\",\\n                           negative_prompt= \\\"text, watermark, lowres, low quality, worst quality, deformed, glitch, low contrast, noisy, saturation, blurry\\\",\\n                           scale=1.0,\\n                           guidance_scale=5,\\n                           num_samples=1,\\n                           num_inference_steps=30, \\n                           seed=42,\\n                           height=2048,\\n                           width=2048\\n                          )\\n```\\n\\n## Distributed Inference\\nOn distributed setups, you can run inference across multiple GPUs with 🤗 Accelerate or PyTorch Distributed, which is useful for generating with multiple prompts in parallel, in case you have limited VRAM on each GPU. More information can be found [here](https://huggingface.co/docs/diffusers/main/en/training/distributed_inference#device-placement). Make sure you have installed diffusers from the source and the lastest accelerate.\\n\\n```python\\nmax_memory = {0:\\\"10GB\\\", 1:\\\"10GB\\\"}\\npipe = StableDiffusionXLPipeline.from_pretrained(\\n    base_model_path,\\n    torch_dtype=torch.float16,\\n    add_watermarker=False,\\n    device_map=\\\"balanced\\\",\\n    max_memory=max_memory\\n)\\n```\\n\\n## Start a local gradio demo <a href='https://github.com/gradio-app/gradio'><img src='https://img.shields.io/github/stars/gradio-app/gradio'></a>\\nRun the following command:\\n```sh\\ngit clone https://github.com/InstantStyle/InstantStyle.git\\ncd ./InstantStyle/gradio_demo/\\npip install -r requirements.txt\\npython app.py\\n```\\n\\n## Resources\\n- [InstantStyle for WebUI](https://github.com/Mikubill/sd-webui-controlnet/discussions/2770)\\n- [InstantStyle for ComfyUI](https://github.com/cubiq/ComfyUI_IPAdapter_plus)\\n- [InstantID](https://github.com/InstantID/InstantID)\\n\\n## Disclaimer\\nThe pretrained checkpoints follow the license in [IP-Adapter](https://github.com/tencent-ailab/IP-Adapter?tab=readme-ov-file#download-models). Users are granted the freedom to create images using this tool, but they are obligated to comply with local laws and utilize it responsibly. The developers will not assume any responsibility for potential misuse by users.\\n\\n## Acknowledgements\\nInstantStyle is developed by the InstantX team and is highly built on [IP-Adapter](https://github.com/tencent-ailab/IP-Adapter), which has been unfairly compared by many other works. We at InstantStyle make IP-Adapter great again. Additionally, we acknowledge [Hu Ye](https://github.com/xiaohu2015) for his valuable discussion.\\n\\n## Star History\\n[![Star History Chart](https://api.star-history.com/svg?repos=InstantStyle/InstantStyle&type=Date)](https://star-history.com/#InstantStyle/InstantStyle&Date)\\n\\n## Cite\\nIf you find InstantStyle useful for your research and applications, please cite us using this BibTeX:\\n\\n```bibtex\\n@article{wang2024instantstyle,\\n  title={InstantStyle-Plus: Style Transfer with Content-Preserving in Text-to-Image Generation},\\n  author={Wang, Haofan and Xing, Peng and Huang, Renyuan and Ai, Hao and Wang, Qixun and Bai, Xu},\\n  journal={arXiv preprint arXiv:2407.00788},\\n  year={2024}\\n}\\n\\n@article{wang2024instantstyle,\\n  title={InstantStyle: Free Lunch towards Style-Preserving in Text-to-Image Generation},\\n  author={Wang, Haofan and Wang, Qixun and Bai, Xu and Qin, Zekui and Chen, Anthony},\\n  journal={arXiv preprint arXiv:2404.02733},\\n  year={2024}\\n}\\n```\\n\\nFor any question, feel free to contact us via haofanwang.ai@gmail.com.\\n\\n\\n'down_blocks.0.attentions.0.transformer_blocks.0.attn1.processor',\\n'down_blocks.0.attentions.0.transformer_blocks.0.attn2.processor', \\n'down_blocks.0.attentions.1.transformer_blocks.0.attn1.processor', \\n'down_blocks.0.attentions.1.transformer_blocks.0.attn2.processor', \\n'down_blocks.1.attentions.0.transformer_blocks.0.attn1.processor', \\n'down_blocks.1.attentions.0.transformer_blocks.0.attn2.processor', \\n'down_blocks.1.attentions.1.transformer_blocks.0.attn1.processor', \\n'down_blocks.1.attentions.1.transformer_blocks.0.attn2.processor', \\n'down_blocks.2.attentions.0.transformer_blocks.0.attn1.processor', \\n'down_blocks.2.attentions.0.transformer_blocks.0.attn2.processor', \\n'down_blocks.2.attentions.1.transformer_blocks.0.attn1.processor', \\n'down_blocks.2.attentions.1.transformer_blocks.0.attn2.processor', \\n'up_blocks.1.attentions.0.transformer_blocks.0.attn1.processor', \\n'up_blocks.1.attentions.0.transformer_blocks.0.attn2.processor', \\n'up_blocks.1.attentions.1.transformer_blocks.0.attn1.processor', \\n'up_blocks.1.attentions.1.transformer_blocks.0.attn2.processor', \\n'up_blocks.1.attentions.2.transformer_blocks.0.attn1.processor', \\n'up_blocks.1.attentions.2.transformer_blocks.0.attn2.processor', \\n'up_blocks.2.attentions.0.transformer_blocks.0.attn1.processor', \\n'up_blocks.2.attentions.0.transformer_blocks.0.attn2.processor', \\n'up_blocks.2.attentions.1.transformer_blocks.0.attn1.processor',\\n'up_blocks.2.attentions.1.transformer_blocks.0.attn2.processor', \\n'up_blocks.2.attentions.2.transformer_blocks.0.attn1.processor', \\n'up_blocks.2.attentions.2.transformer_blocks.0.attn2.processor', \\n'up_blocks.3.attentions.0.transformer_blocks.0.attn1.processor', \\n'up_blocks.3.attentions.0.transformer_blocks.0.attn2.processor', \\n'up_blocks.3.attentions.1.transformer_blocks.0.attn1.processor', \\n'up_blocks.3.attentions.1.transformer_blocks.0.attn2.processor', \\n'up_blocks.3.attentions.2.transformer_blocks.0.attn1.processor', \\n'up_blocks.3.attentions.2.transformer_blocks.0.attn2.processor', \\n'mid_block.attentions.0.transformer_blocks.0.attn1.processor', \\n'mid_block.attentions.0.transformer_blocks.0.attn2.processor'\\n\\nimport torch\\nfrom diffusers import StableDiffusionXLPipeline\\nfrom PIL import Image\\n\\nfrom ip_adapter import IPAdapterXL\\n\\nbase_model_path = \\\"stabilityai/stable-diffusion-xl-base-1.0\\\"\\nimage_encoder_path = \\\"sdxl_models/image_encoder\\\"\\nip_ckpt = \\\"sdxl_models/ip-adapter_sdxl.bin\\\"\\ndevice = \\\"cuda\\\"\\n\\n# load SDXL pipeline\\npipe = StableDiffusionXLPipeline.from_pretrained(\\n    base_model_path,\\n    torch_dtype=torch.float16,\\n    add_watermarker=False,\\n)\\npipe.enable_vae_tiling()\\n\\n# load ip-adapter\\n# target_blocks=[\\\"block\\\"] for original IP-Adapter\\n# target_blocks=[\\\"up_blocks.0.attentions.1\\\"] for style blocks only\\n# target_blocks = [\\\"up_blocks.0.attentions.1\\\", \\\"down_blocks.2.attentions.1\\\"] # for style+layout blocks\\nip_model = IPAdapterXL(pipe, image_encoder_path, ip_ckpt, device, target_blocks=[\\\"up_blocks.0.attentions.1\\\"])\\n\\nimage = \\\"./assets/0.jpg\\\"\\nimage = Image.open(image)\\nimage.resize((512, 512))\\n\\n# generate image\\nimages = ip_model.generate(pil_image=image,\\n                           prompt=\\\"a cat, masterpiece, best quality, high quality\\\",\\n                           negative_prompt= \\\"text, watermark, lowres, low quality, worst quality, deformed, glitch, low contrast, noisy, saturation, blurry\\\",\\n                           scale=1.0,\\n                           guidance_scale=5,\\n                           num_samples=1,\\n                           num_inference_steps=30, \\n                           seed=42,\\n                           #neg_content_prompt=\\\"a rabbit\\\",\\n                           #neg_content_scale=0.5,\\n                          )\\n\\nimages[0].save(\\\"result.png\\\")\\n\\ntitle: InstantStyle\\nemoji: 👁\\ncolorFrom: blue\\ncolorTo: purple\\nsdk: gradio\\nsdk_version: 4.26.0\\napp_file: app.py\\npinned: false\\nlicense: apache-2.0\\n\\ndiffusers>=0.25.1\\ntorch>=2.0.0\\ntorchvision>=0.15.1\\ntransformers>=4.37.1\\naccelerate\\nsafetensors\\neinops\\nspaces>=0.19.4\\nomegaconf\\npeft\\nhuggingface-hub>=0.20.2\\nopencv-python\\ngradio\\ncontrolnet_aux\\ngdown\\npeft\\n\\nimport sys\\nsys.path.append('../')\\n\\nimport os \\nimport cv2\\nimport torch\\nimport random\\nimport numpy as np\\nfrom PIL import Image\\nfrom diffusers import ControlNetModel, StableDiffusionXLControlNetPipeline\\n\\nimport gradio as gr\\n\\nfrom ip_adapter import IPAdapterXL\\n\\n# global variable\\nMAX_SEED = np.iinfo(np.int32).max\\ndevice = \\\"cuda\\\" if torch.cuda.is_available() else \\\"cpu\\\"\\ndtype = torch.float16 if str(device).__contains__(\\\"cuda\\\") else torch.float32\\n\\n# initialization\\nbase_model_path = \\\"stabilityai/stable-diffusion-xl-base-1.0\\\"\\nimage_encoder_path = \\\"sdxl_models/image_encoder\\\"\\nip_ckpt = \\\"sdxl_models/ip-adapter_sdxl.bin\\\"\\n\\ncontrolnet_path = \\\"diffusers/controlnet-canny-sdxl-1.0\\\"\\ncontrolnet = ControlNetModel.from_pretrained(controlnet_path, use_safetensors=False, torch_dtype=torch.float16).to(device)\\n\\n# load SDXL pipeline\\npipe = StableDiffusionXLControlNetPipeline.from_pretrained(\\n    base_model_path,\\n    controlnet=controlnet,\\n    torch_dtype=torch.float16,\\n    add_watermarker=False,\\n)\\npipe.enable_vae_tiling()\\n\\n# load ip-adapter\\n# target_blocks=[\\\"block\\\"] for original IP-Adapter\\n# target_blocks=[\\\"up_blocks.0.attentions.1\\\"] for style blocks only\\n# target_blocks = [\\\"up_blocks.0.attentions.1\\\", \\\"down_blocks.2.attentions.1\\\"] # for style+layout blocks\\nip_model = IPAdapterXL(pipe, image_encoder_path, ip_ckpt, device, target_blocks=[\\\"up_blocks.0.attentions.1\\\"])\\n\\ndef randomize_seed_fn(seed: int, randomize_seed: bool) -> int:\\n    if randomize_seed:\\n        seed = random.randint(0, MAX_SEED)\\n    return seed\\n\\ndef resize_img(\\n    input_image,\\n    max_side=1280,\\n    min_side=1024,\\n    size=None,\\n    pad_to_max_side=False,\\n    mode=Image.BILINEAR,\\n    base_pixel_number=64,\\n):\\n    w, h = input_image.size\\n    if size is not None:\\n        w_resize_new, h_resize_new = size\\n    else:\\n        ratio = min_side / min(h, w)\\n        w, h = round(ratio * w), round(ratio * h)\\n        ratio = max_side / max(h, w)\\n        input_image = input_image.resize([round(ratio * w), round(ratio * h)], mode)\\n        w_resize_new = (round(ratio * w) // base_pixel_number) * base_pixel_number\\n        h_resize_new = (round(ratio * h) // base_pixel_number) * base_pixel_number\\n    input_image = input_image.resize([w_resize_new, h_resize_new], mode)\\n\\n    if pad_to_max_side:\\n        res = np.ones([max_side, max_side, 3], dtype=np.uint8) * 255\\n        offset_x = (max_side - w_resize_new) // 2\\n        offset_y = (max_side - h_resize_new) // 2\\n        res[\\n            offset_y : offset_y + h_resize_new, offset_x : offset_x + w_resize_new\\n        ] = np.array(input_image)\\n        input_image = Image.fromarray(res)\\n    return input_image\\n\\ndef get_example():\\n    case = [\\n        [\\n            \\\"./assets/0.jpg\\\",\\n            None,\\n            \\\"a cat, masterpiece, best quality, high quality\\\",\\n            1.0,\\n            0.0\\n        ],\\n        [\\n            \\\"./assets/1.jpg\\\",\\n            None,\\n            \\\"a cat, masterpiece, best quality, high quality\\\",\\n            1.0,\\n            0.0\\n        ],\\n        [\\n            \\\"./assets/2.jpg\\\",\\n            None,\\n            \\\"a cat, masterpiece, best quality, high quality\\\",\\n            1.0,\\n            0.0\\n        ],\\n        [\\n            \\\"./assets/3.jpg\\\",\\n            None,\\n            \\\"a cat, masterpiece, best quality, high quality\\\",\\n            1.0,\\n            0.0\\n        ],\\n        [\\n            \\\"./assets/2.jpg\\\",\\n            \\\"./assets/yann-lecun.jpg\\\",\\n            \\\"a man, masterpiece, best quality, high quality\\\",\\n            1.0,\\n            0.6\\n        ],\\n    ]\\n    return case\\n\\ndef run_for_examples(style_image, source_image, prompt, scale, control_scale):\\n\\n    return create_image(\\n        image_pil=style_image,\\n        input_image=source_image,\\n        prompt=prompt,\\n        n_prompt=\\\"text, watermark, lowres, low quality, worst quality, deformed, glitch, low contrast, noisy, saturation, blurry\\\",\\n        scale=scale,\\n        control_scale=control_scale,\\n        guidance_scale=5,\\n        num_samples=1,\\n        num_inference_steps=20,\\n        seed=42,\\n        target=\\\"Load only style blocks\\\",\\n        neg_content_prompt=\\\"\\\",\\n        neg_content_scale=0,\\n    )\\n\\ndef create_image(image_pil,\\n                 input_image,\\n                 prompt,\\n                 n_prompt,\\n                 scale, \\n                 control_scale, \\n                 guidance_scale,\\n                 num_samples,\\n                 num_inference_steps,\\n                 seed,\\n                 target=\\\"Load only style blocks\\\",\\n                 neg_content_prompt=None,\\n                 neg_content_scale=0):\\n\\n    if target ==\\\"Load original IP-Adapter\\\":\\n        # target_blocks=[\\\"blocks\\\"] for original IP-Adapter\\n        ip_model = IPAdapterXL(pipe, image_encoder_path, ip_ckpt, device, target_blocks=[\\\"blocks\\\"])\\n    elif target==\\\"Load only style blocks\\\":\\n        # target_blocks=[\\\"up_blocks.0.attentions.1\\\"] for style blocks only\\n        ip_model = IPAdapterXL(pipe, image_encoder_path, ip_ckpt, device, target_blocks=[\\\"up_blocks.0.attentions.1\\\"])\\n    elif target == \\\"Load style+layout block\\\":\\n        # target_blocks = [\\\"up_blocks.0.attentions.1\\\", \\\"down_blocks.2.attentions.1\\\"] # for style+layout blocks\\n        ip_model = IPAdapterXL(pipe, image_encoder_path, ip_ckpt, device, target_blocks=[\\\"up_blocks.0.attentions.1\\\", \\\"down_blocks.2.attentions.1\\\"])\\n    \\n    if input_image is not None:\\n        input_image = resize_img(input_image, max_side=1024)\\n        cv_input_image = pil_to_cv2(input_image)\\n        detected_map = cv2.Canny(cv_input_image, 50, 200)\\n        canny_map = Image.fromarray(cv2.cvtColor(detected_map, cv2.COLOR_BGR2RGB))\\n    else:\\n        canny_map = Image.new('RGB', (1024, 1024), color=(255, 255, 255))\\n        control_scale = 0\\n\\n    if float(control_scale) == 0:\\n        canny_map = canny_map.resize((1024,1024))\\n    \\n    if len(neg_content_prompt) > 0 and neg_content_scale != 0:\\n        images = ip_model.generate(pil_image=image_pil,\\n                                prompt=prompt,\\n                                negative_prompt=n_prompt,\\n                                scale=scale,\\n                                guidance_scale=guidance_scale,\\n                                num_samples=num_samples,\\n                                num_inference_steps=num_inference_steps, \\n                                seed=seed,\\n                                image=canny_map,\\n                                controlnet_conditioning_scale=float(control_scale),\\n                                neg_content_prompt=neg_content_prompt,\\n                                neg_content_scale=neg_content_scale\\n                                )\\n    else:\\n        images = ip_model.generate(pil_image=image_pil,\\n                                prompt=prompt,\\n                                negative_prompt=n_prompt,\\n                                scale=scale,\\n                                guidance_scale=guidance_scale,\\n                                num_samples=num_samples,\\n                                num_inference_steps=num_inference_steps, \\n                                seed=seed,\\n                                image=canny_map,\\n                                controlnet_conditioning_scale=float(control_scale),\\n                                )\\n    return images\\n\\ndef pil_to_cv2(image_pil):\\n    image_np = np.array(image_pil)\\n    image_cv2 = cv2.cvtColor(image_np, cv2.COLOR_RGB2BGR)\\n    return image_cv2\\n\\n# Description\\ntitle = r\\\"\\\"\\\"\\n<h1 align=\\\"center\\\">InstantStyle: Free Lunch towards Style-Preserving in Text-to-Image Generation</h1>\\n\\\"\\\"\\\"\\n\\ndescription = r\\\"\\\"\\\"\\n<b>Official 🤗 Gradio demo</b> for <a href='https://github.com/InstantStyle/InstantStyle' target='_blank'><b>InstantStyle: Free Lunch towards Style-Preserving in Text-to-Image Generation</b></a>.<br>\\nHow to use:<br>\\n1. Upload a style image.\\n2. Set stylization mode, only use style block by default.\\n2. Enter a text prompt, as done in normal text-to-image models.\\n3. Click the <b>Submit</b> button to begin customization.\\n4. Share your stylized photo with your friends and enjoy! 😊\\nAdvanced usage:<br>\\n1. Click advanced options.\\n2. Upload another source image for image-based stylization using ControlNet.\\n3. Enter negative content prompt to avoid content leakage.\\n\\\"\\\"\\\"\\n\\narticle = r\\\"\\\"\\\"\\n---\\n📝 **Citation**\\n<br>\\nIf our work is helpful for your research or applications, please cite us via:\\n```bibtex\\n@article{wang2024instantstyle,\\n  title={InstantStyle: Free Lunch towards Style-Preserving in Text-to-Image Generation},\\n  author={Wang, Haofan and Wang, Qixun and Bai, Xu and Qin, Zekui and Chen, Anthony},\\n  journal={arXiv preprint arXiv:2404.02733},\\n  year={2024}\\n}\\n```\\n📧 **Contact**\\n<br>\\nIf you have any questions, please feel free to open an issue or directly reach us out at <b>haofanwang.ai@gmail.com</b>.\\n\\\"\\\"\\\"\\n\\nblock = gr.Blocks(css=\\\"footer {visibility: hidden}\\\").queue(max_size=10, api_open=False)\\nwith block:\\n    \\n    # description\\n    gr.Markdown(title)\\n    gr.Markdown(description)\\n    \\n    with gr.Tabs():\\n        with gr.Row():\\n            with gr.Column():\\n                \\n                with gr.Row():\\n                    with gr.Column():\\n                        image_pil = gr.Image(label=\\\"Style Image\\\", type='pil')\\n                \\n                target = gr.Radio([\\\"Load only style blocks\\\", \\\"Load style+layout block\\\", \\\"Load original IP-Adapter\\\"], \\n                                  value=\\\"Load only style blocks\\\",\\n                                  label=\\\"Style mode\\\")\\n                \\n                prompt = gr.Textbox(label=\\\"Prompt\\\",\\n                                    value=\\\"a cat, masterpiece, best quality, high quality\\\")\\n                \\n                scale = gr.Slider(minimum=0,maximum=2.0, step=0.01,value=1.0, label=\\\"Scale\\\")\\n                \\n                with gr.Accordion(open=False, label=\\\"Advanced Options\\\"):\\n                    \\n                    with gr.Column():\\n                        src_image_pil = gr.Image(label=\\\"Source Image (optional)\\\", type='pil')\\n                    control_scale = gr.Slider(minimum=0,maximum=1.0, step=0.01,value=0.5, label=\\\"Controlnet conditioning scale\\\")\\n                    \\n                    n_prompt = gr.Textbox(label=\\\"Neg Prompt\\\", value=\\\"text, watermark, lowres, low quality, worst quality, deformed, glitch, low contrast, noisy, saturation, blurry\\\")\\n                    \\n                    neg_content_prompt = gr.Textbox(label=\\\"Neg Content Prompt\\\", value=\\\"\\\")\\n                    neg_content_scale = gr.Slider(minimum=0, maximum=1.0, step=0.01,value=0.5, label=\\\"Neg Content Scale\\\")\\n\\n                    guidance_scale = gr.Slider(minimum=1,maximum=15.0, step=0.01,value=5.0, label=\\\"guidance scale\\\")\\n                    num_samples= gr.Slider(minimum=1,maximum=4.0, step=1.0,value=1.0, label=\\\"num samples\\\")\\n                    num_inference_steps = gr.Slider(minimum=5,maximum=50.0, step=1.0,value=20, label=\\\"num inference steps\\\")\\n                    seed = gr.Slider(minimum=-1000000,maximum=1000000,value=1, step=1, label=\\\"Seed Value\\\")\\n                    randomize_seed = gr.Checkbox(label=\\\"Randomize seed\\\", value=True)\\n                    \\n                generate_button = gr.Button(\\\"Generate Image\\\")\\n                \\n            with gr.Column():\\n                generated_image = gr.Gallery(label=\\\"Generated Image\\\")\\n\\n        generate_button.click(\\n            fn=randomize_seed_fn,\\n            inputs=[seed, randomize_seed],\\n            outputs=seed,\\n            queue=False,\\n            api_name=False,\\n        ).then(\\n            fn=create_image,\\n            inputs=[image_pil,\\n                    src_image_pil,\\n                    prompt,\\n                    n_prompt,\\n                    scale, \\n                    control_scale, \\n                    guidance_scale,\\n                    num_samples,\\n                    num_inference_steps,\\n                    seed,\\n                    target,\\n                    neg_content_prompt,\\n                    neg_content_scale], \\n            outputs=[generated_image])\\n    \\n    gr.Examples(\\n        examples=get_example(),\\n        inputs=[image_pil, src_image_pil, prompt, scale, control_scale],\\n        fn=run_for_examples,\\n        outputs=[generated_image],\\n        cache_examples=True,\\n    )\\n    \\n    gr.Markdown(article)\\n\\nblock.launch()\\n\\n\\n# modified from https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/attention_processor.py\\nimport torch\\nimport torch.nn as nn\\nimport torch.nn.functional as F\\n\\n\\nclass AttnProcessor(nn.Module):\\n    r\\\"\\\"\\\"\\n    Default processor for performing attention-related computations.\\n    \\\"\\\"\\\"\\n\\n    def __init__(\\n        self,\\n        hidden_size=None,\\n        cross_attention_dim=None,\\n    ):\\n        super().__init__()\\n\\n    def __call__(\\n        self,\\n        attn,\\n        hidden_states,\\n        encoder_hidden_states=None,\\n        attention_mask=None,\\n        temb=None,\\n    ):\\n        residual = hidden_states\\n\\n        if attn.spatial_norm is not None:\\n            hidden_states = attn.spatial_norm(hidden_states, temb)\\n\\n        input_ndim = hidden_states.ndim\\n\\n        if input_ndim == 4:\\n            batch_size, channel, height, width = hidden_states.shape\\n            hidden_states = hidden_states.view(batch_size, channel, height * width).transpose(1, 2)\\n\\n        batch_size, sequence_length, _ = (\\n            hidden_states.shape if encoder_hidden_states is None else encoder_hidden_states.shape\\n        )\\n        attention_mask = attn.prepare_attention_mask(attention_mask, sequence_length, batch_size)\\n\\n        if attn.group_norm is not None:\\n            hidden_states = attn.group_norm(hidden_states.transpose(1, 2)).transpose(1, 2)\\n\\n        query = attn.to_q(hidden_states)\\n\\n        if encoder_hidden_states is None:\\n            encoder_hidden_states = hidden_states\\n        elif attn.norm_cross:\\n            encoder_hidden_states = attn.norm_encoder_hidden_states(encoder_hidden_states)\\n\\n        key = attn.to_k(encoder_hidden_states)\\n        value = attn.to_v(encoder_hidden_states)\\n\\n        query = attn.head_to_batch_dim(query)\\n        key = attn.head_to_batch_dim(key)\\n        value = attn.head_to_batch_dim(value)\\n\\n        attention_probs = attn.get_attention_scores(query, key, attention_mask)\\n        hidden_states = torch.bmm(attention_probs, value)\\n        hidden_states = attn.batch_to_head_dim(hidden_states)\\n\\n        # linear proj\\n        hidden_states = attn.to_out[0](hidden_states)\\n        # dropout\\n        hidden_states = attn.to_out[1](hidden_states)\\n\\n        if input_ndim == 4:\\n            hidden_states = hidden_states.transpose(-1, -2).reshape(batch_size, channel, height, width)\\n\\n        if attn.residual_connection:\\n            hidden_states = hidden_states + residual\\n\\n        hidden_states = hidden_states / attn.rescale_output_factor\\n\\n        return hidden_states\\n\\n\\nclass IPAttnProcessor(nn.Module):\\n    r\\\"\\\"\\\"\\n    Attention processor for IP-Adapater.\\n    Args:\\n        hidden_size (`int`):\\n            The hidden size of the attention layer.\\n        cross_attention_dim (`int`):\\n            The number of channels in the `encoder_hidden_states`.\\n        scale (`float`, defaults to 1.0):\\n            the weight scale of image prompt.\\n        num_tokens (`int`, defaults to 4 when do ip_adapter_plus it should be 16):\\n            The context length of the image features.\\n    \\\"\\\"\\\"\\n\\n    def __init__(self, hidden_size, cross_attention_dim=None, scale=1.0, num_tokens=4, skip=False):\\n        super().__init__()\\n\\n        self.hidden_size = hidden_size\\n        self.cross_attention_dim = cross_attention_dim\\n        self.scale = scale\\n        self.num_tokens = num_tokens\\n        self.skip = skip\\n\\n        self.to_k_ip = nn.Linear(cross_attention_dim or hidden_size, hidden_size, bias=False)\\n        self.to_v_ip = nn.Linear(cross_attention_dim or hidden_size, hidden_size, bias=False)\\n\\n    def __call__(\\n        self,\\n        attn,\\n        hidden_states,\\n        encoder_hidden_states=None,\\n        attention_mask=None,\\n        temb=None,\\n    ):\\n        residual = hidden_states\\n\\n        if attn.spatial_norm is not None:\\n            hidden_states = attn.spatial_norm(hidden_states, temb)\\n\\n        input_ndim = hidden_states.ndim\\n\\n        if input_ndim == 4:\\n            batch_size, channel, height, width = hidden_states.shape\\n            hidden_states = hidden_states.view(batch_size, channel, height * width).transpose(1, 2)\\n\\n        batch_size, sequence_length, _ = (\\n            hidden_states.shape if encoder_hidden_states is None else encoder_hidden_states.shape\\n        )\\n        attention_mask = attn.prepare_attention_mask(attention_mask, sequence_length, batch_size)\\n\\n        if attn.group_norm is not None:\\n            hidden_states = attn.group_norm(hidden_states.transpose(1, 2)).transpose(1, 2)\\n\\n        query = attn.to_q(hidden_states)\\n\\n        if encoder_hidden_states is None:\\n            encoder_hidden_states = hidden_states\\n        else:\\n            # get encoder_hidden_states, ip_hidden_states\\n            end_pos = encoder_hidden_states.shape[1] - self.num_tokens\\n            encoder_hidden_states, ip_hidden_states = (\\n                encoder_hidden_states[:, :end_pos, :],\\n                encoder_hidden_states[:, end_pos:, :],\\n            )\\n            if attn.norm_cross:\\n                encoder_hidden_states = attn.norm_encoder_hidden_states(encoder_hidden_states)\\n\\n        key = attn.to_k(encoder_hidden_states)\\n        value = attn.to_v(encoder_hidden_states)\\n\\n        query = attn.head_to_batch_dim(query)\\n        key = attn.head_to_batch_dim(key)\\n        value = attn.head_to_batch_dim(value)\\n\\n        attention_probs = attn.get_attention_scores(query, key, attention_mask)\\n        hidden_states = torch.bmm(attention_probs, value)\\n        hidden_states = attn.batch_to_head_dim(hidden_states)\\n\\n        if not self.skip:\\n            # for ip-adapter\\n            ip_key = self.to_k_ip(ip_hidden_states)\\n            ip_value = self.to_v_ip(ip_hidden_states)\\n\\n            ip_key = attn.head_to_batch_dim(ip_key)\\n            ip_value = attn.head_to_batch_dim(ip_value)\\n\\n            ip_attention_probs = attn.get_attention_scores(query, ip_key, None)\\n            self.attn_map = ip_attention_probs\\n            ip_hidden_states = torch.bmm(ip_attention_probs, ip_value)\\n            ip_hidden_states = attn.batch_to_head_dim(ip_hidden_states)\\n\\n            hidden_states = hidden_states + self.scale * ip_hidden_states\\n\\n        # linear proj\\n        hidden_states = attn.to_out[0](hidden_states)\\n        # dropout\\n        hidden_states = attn.to_out[1](hidden_states)\\n\\n        if input_ndim == 4:\\n            hidden_states = hidden_states.transpose(-1, -2).reshape(batch_size, channel, height, width)\\n\\n        if attn.residual_connection:\\n            hidden_states = hidden_states + residual\\n\\n        hidden_states = hidden_states / attn.rescale_output_factor\\n\\n        return hidden_states\\n\\n\\nclass AttnProcessor2_0(torch.nn.Module):\\n    r\\\"\\\"\\\"\\n    Processor for implementing scaled dot-product attention (enabled by default if you're using PyTorch 2.0).\\n    \\\"\\\"\\\"\\n\\n    def __init__(\\n        self,\\n        hidden_size=None,\\n        cross_attention_dim=None,\\n    ):\\n        super().__init__()\\n        if not hasattr(F, \\\"scaled_dot_product_attention\\\"):\\n            raise ImportError(\\\"AttnProcessor2_0 requires PyTorch 2.0, to use it, please upgrade PyTorch to 2.0.\\\")\\n\\n    def __call__(\\n        self,\\n        attn,\\n        hidden_states,\\n        encoder_hidden_states=None,\\n        attention_mask=None,\\n        temb=None,\\n    ):\\n        residual = hidden_states\\n\\n        if attn.spatial_norm is not None:\\n            hidden_states = attn.spatial_norm(hidden_states, temb)\\n\\n        input_ndim = hidden_states.ndim\\n\\n        if input_ndim == 4:\\n            batch_size, channel, height, width = hidden_states.shape\\n            hidden_states = hidden_states.view(batch_size, channel, height * width).transpose(1, 2)\\n\\n        batch_size, sequence_length, _ = (\\n            hidden_states.shape if encoder_hidden_states is None else encoder_hidden_states.shape\\n        )\\n\\n        if attention_mask is not None:\\n            attention_mask = attn.prepare_attention_mask(attention_mask, sequence_length, batch_size)\\n            # scaled_dot_product_attention expects attention_mask shape to be\\n            # (batch, heads, source_length, target_length)\\n            attention_mask = attention_mask.view(batch_size, attn.heads, -1, attention_mask.shape[-1])\\n\\n        if attn.group_norm is not None:\\n            hidden_states = attn.group_norm(hidden_states.transpose(1, 2)).transpose(1, 2)\\n\\n        query = attn.to_q(hidden_states)\\n\\n        if encoder_hidden_states is None:\\n            encoder_hidden_states = hidden_states\\n        elif attn.norm_cross:\\n            encoder_hidden_states = attn.norm_encoder_hidden_states(encoder_hidden_states)\\n\\n        key = attn.to_k(encoder_hidden_states)\\n        value = attn.to_v(encoder_hidden_states)\\n\\n        inner_dim = key.shape[-1]\\n        head_dim = inner_dim // attn.heads\\n\\n        query = query.view(batch_size, -1, attn.heads, head_dim).transpose(1, 2)\\n\\n        key = key.view(batch_size, -1, attn.heads, head_dim).transpose(1, 2)\\n        value = value.view(batch_size, -1, attn.heads, head_dim).transpose(1, 2)\\n\\n        # the output of sdp = (batch, num_heads, seq_len, head_dim)\\n        # TODO: add support for attn.scale when we move to Torch 2.1\\n        hidden_states = F.scaled_dot_product_attention(\\n            query, key, value, attn_mask=attention_mask, dropout_p=0.0, is_causal=False\\n        )\\n\\n        hidden_states = hidden_states.transpose(1, 2).reshape(batch_size, -1, attn.heads * head_dim)\\n        hidden_states = hidden_states.to(query.dtype)\\n\\n        # linear proj\\n        hidden_states = attn.to_out[0](hidden_states)\\n        # dropout\\n        hidden_states = attn.to_out[1](hidden_states)\\n\\n        if input_ndim == 4:\\n            hidden_states = hidden_states.transpose(-1, -2).reshape(batch_size, channel, height, width)\\n\\n        if attn.residual_connection:\\n            hidden_states = hidden_states + residual\\n\\n        hidden_states = hidden_states / attn.rescale_output_factor\\n\\n        return hidden_states\\n\\n\\nclass IPAttnProcessor2_0(torch.nn.Module):\\n    r\\\"\\\"\\\"\\n    Attention processor for IP-Adapater for PyTorch 2.0.\\n    Args:\\n        hidden_size (`int`):\\n            The hidden size of the attention layer.\\n        cross_attention_dim (`int`):\\n            The number of channels in the `encoder_hidden_states`.\\n        scale (`float`, defaults to 1.0):\\n            the weight scale of image prompt.\\n        num_tokens (`int`, defaults to 4 when do ip_adapter_plus it should be 16):\\n            The context length of the image features.\\n    \\\"\\\"\\\"\\n\\n    def __init__(self, hidden_size, cross_attention_dim=None, scale=1.0, num_tokens=4, skip=False):\\n        super().__init__()\\n\\n        if not hasattr(F, \\\"scaled_dot_product_attention\\\"):\\n            raise ImportError(\\\"AttnProcessor2_0 requires PyTorch 2.0, to use it, please upgrade PyTorch to 2.0.\\\")\\n\\n        self.hidden_size = hidden_size\\n        self.cross_attention_dim = cross_attention_dim\\n        self.scale = scale\\n        self.num_tokens = num_tokens\\n        self.skip = skip\\n\\n        self.to_k_ip = nn.Linear(cross_attention_dim or hidden_size, hidden_size, bias=False)\\n        self.to_v_ip = nn.Linear(cross_attention_dim or hidden_size, hidden_size, bias=False)\\n\\n    def __call__(\\n        self,\\n        attn,\\n        hidden_states,\\n        encoder_hidden_states=None,\\n        attention_mask=None,\\n        temb=None,\\n    ):\\n        residual = hidden_states\\n\\n        if attn.spatial_norm is not None:\\n            hidden_states = attn.spatial_norm(hidden_states, temb)\\n\\n        input_ndim = hidden_states.ndim\\n\\n        if input_ndim == 4:\\n            batch_size, channel, height, width = hidden_states.shape\\n            hidden_states = hidden_states.view(batch_size, channel, height * width).transpose(1, 2)\\n\\n        batch_size, sequence_length, _ = (\\n            hidden_states.shape if encoder_hidden_states is None else encoder_hidden_states.shape\\n        )\\n\\n        if attention_mask is not None:\\n            attention_mask = attn.prepare_attention_mask(attention_mask, sequence_length, batch_size)\\n            # scaled_dot_product_attention expects attention_mask shape to be\\n            # (batch, heads, source_length, target_length)\\n            attention_mask = attention_mask.view(batch_size, attn.heads, -1, attention_mask.shape[-1])\\n\\n        if attn.group_norm is not None:\\n            hidden_states = attn.group_norm(hidden_states.transpose(1, 2)).transpose(1, 2)\\n\\n        query = attn.to_q(hidden_states)\\n\\n        if encoder_hidden_states is None:\\n            encoder_hidden_states = hidden_states\\n        else:\\n            # get encoder_hidden_states, ip_hidden_states\\n            end_pos = encoder_hidden_states.shape[1] - self.num_tokens\\n            encoder_hidden_states, ip_hidden_states = (\\n                encoder_hidden_states[:, :end_pos, :],\\n                encoder_hidden_states[:, end_pos:, :],\\n            )\\n            if attn.norm_cross:\\n                encoder_hidden_states = attn.norm_encoder_hidden_states(encoder_hidden_states)\\n\\n        key = attn.to_k(encoder_hidden_states)\\n        value = attn.to_v(encoder_hidden_states)\\n\\n        inner_dim = key.shape[-1]\\n        head_dim = inner_dim // attn.heads\\n\\n        query = query.view(batch_size, -1, attn.heads, head_dim).transpose(1, 2)\\n\\n        key = key.view(batch_size, -1, attn.heads, head_dim).transpose(1, 2)\\n        value = value.view(batch_size, -1, attn.heads, head_dim).transpose(1, 2)\\n\\n        # the output of sdp = (batch, num_heads, seq_len, head_dim)\\n        # TODO: add support for attn.scale when we move to Torch 2.1\\n        hidden_states = F.scaled_dot_product_attention(\\n            query, key, value, attn_mask=attention_mask, dropout_p=0.0, is_causal=False\\n        )\\n\\n        hidden_states = hidden_states.transpose(1, 2).reshape(batch_size, -1, attn.heads * head_dim)\\n        hidden_states = hidden_states.to(query.dtype)\\n\\n        if not self.skip:\\n            # for ip-adapter\\n            ip_key = self.to_k_ip(ip_hidden_states)\\n            ip_value = self.to_v_ip(ip_hidden_states)\\n\\n            ip_key = ip_key.view(batch_size, -1, attn.heads, head_dim).transpose(1, 2)\\n            ip_value = ip_value.view(batch_size, -1, attn.heads, head_dim).transpose(1, 2)\\n\\n            # the output of sdp = (batch, num_heads, seq_len, head_dim)\\n            # TODO: add support for attn.scale when we move to Torch 2.1\\n            ip_hidden_states = F.scaled_dot_product_attention(\\n                query, ip_key, ip_value, attn_mask=None, dropout_p=0.0, is_causal=False\\n            )\\n            with torch.no_grad():\\n                self.attn_map = query @ ip_key.transpose(-2, -1).softmax(dim=-1)\\n                #print(self.attn_map.shape)\\n\\n            ip_hidden_states = ip_hidden_states.transpose(1, 2).reshape(batch_size, -1, attn.heads * head_dim)\\n            ip_hidden_states = ip_hidden_states.to(query.dtype)\\n\\n            hidden_states = hidden_states + self.scale * ip_hidden_states\\n\\n        # linear proj\\n        hidden_states = attn.to_out[0](hidden_states)\\n        # dropout\\n        hidden_states = attn.to_out[1](hidden_states)\\n\\n        if input_ndim == 4:\\n            hidden_states = hidden_states.transpose(-1, -2).reshape(batch_size, channel, height, width)\\n\\n        if attn.residual_connection:\\n            hidden_states = hidden_states + residual\\n\\n        hidden_states = hidden_states / attn.rescale_output_factor\\n\\n        return hidden_states\\n\\n\\n## for controlnet\\nclass CNAttnProcessor:\\n    r\\\"\\\"\\\"\\n    Default processor for performing attention-related computations.\\n    \\\"\\\"\\\"\\n\\n    def __init__(self, num_tokens=4):\\n        self.num_tokens = num_tokens\\n\\n    def __call__(self, attn, hidden_states, encoder_hidden_states=None, attention_mask=None, temb=None):\\n        residual = hidden_states\\n\\n        if attn.spatial_norm is not None:\\n            hidden_states = attn.spatial_norm(hidden_states, temb)\\n\\n        input_ndim = hidden_states.ndim\\n\\n        if input_ndim == 4:\\n            batch_size, channel, height, width = hidden_states.shape\\n            hidden_states = hidden_states.view(batch_size, channel, height * width).transpose(1, 2)\\n\\n        batch_size, sequence_length, _ = (\\n            hidden_states.shape if encoder_hidden_states is None else encoder_hidden_states.shape\\n        )\\n        attention_mask = attn.prepare_attention_mask(attention_mask, sequence_length, batch_size)\\n\\n        if attn.group_norm is not None:\\n            hidden_states = attn.group_norm(hidden_states.transpose(1, 2)).transpose(1, 2)\\n\\n        query = attn.to_q(hidden_states)\\n\\n        if encoder_hidden_states is None:\\n            encoder_hidden_states = hidden_states\\n        else:\\n            end_pos = encoder_hidden_states.shape[1] - self.num_tokens\\n            encoder_hidden_states = encoder_hidden_states[:, :end_pos]  # only use text\\n            if attn.norm_cross:\\n                encoder_hidden_states = attn.norm_encoder_hidden_states(encoder_hidden_states)\\n\\n        key = attn.to_k(encoder_hidden_states)\\n        value = attn.to_v(encoder_hidden_states)\\n\\n        query = attn.head_to_batch_dim(query)\\n        key = attn.head_to_batch_dim(key)\\n        value = attn.head_to_batch_dim(value)\\n\\n        attention_probs = attn.get_attention_scores(query, key, attention_mask)\\n        hidden_states = torch.bmm(attention_probs, value)\\n        hidden_states = attn.batch_to_head_dim(hidden_states)\\n\\n        # linear proj\\n        hidden_states = attn.to_out[0](hidden_states)\\n        # dropout\\n        hidden_states = attn.to_out[1](hidden_states)\\n\\n        if input_ndim == 4:\\n            hidden_states = hidden_states.transpose(-1, -2).reshape(batch_size, channel, height, width)\\n\\n        if attn.residual_connection:\\n            hidden_states = hidden_states + residual\\n\\n        hidden_states = hidden_states / attn.rescale_output_factor\\n\\n        return hidden_states\\n\\n\\nclass CNAttnProcessor2_0:\\n    r\\\"\\\"\\\"\\n    Processor for implementing scaled dot-product attention (enabled by default if you're using PyTorch 2.0).\\n    \\\"\\\"\\\"\\n\\n    def __init__(self, num_tokens=4):\\n        if not hasattr(F, \\\"scaled_dot_product_attention\\\"):\\n            raise ImportError(\\\"AttnProcessor2_0 requires PyTorch 2.0, to use it, please upgrade PyTorch to 2.0.\\\")\\n        self.num_tokens = num_tokens\\n\\n    def __call__(\\n        self,\\n        attn,\\n        hidden_states,\\n        encoder_hidden_states=None,\\n        attention_mask=None,\\n        temb=None,\\n    ):\\n        residual = hidden_states\\n\\n        if attn.spatial_norm is not None:\\n            hidden_states = attn.spatial_norm(hidden_states, temb)\\n\\n        input_ndim = hidden_states.ndim\\n\\n        if input_ndim == 4:\\n            batch_size, channel, height, width = hidden_states.shape\\n            hidden_states = hidden_states.view(batch_size, channel, height * width).transpose(1, 2)\\n\\n        batch_size, sequence_length, _ = (\\n            hidden_states.shape if encoder_hidden_states is None else encoder_hidden_states.shape\\n        )\\n\\n        if attention_mask is not None:\\n            attention_mask = attn.prepare_attention_mask(attention_mask, sequence_length, batch_size)\\n            # scaled_dot_product_attention expects attention_mask shape to be\\n            # (batch, heads, source_length, target_length)\\n            attention_mask = attention_mask.view(batch_size, attn.heads, -1, attention_mask.shape[-1])\\n\\n        if attn.group_norm is not None:\\n            hidden_states = attn.group_norm(hidden_states.transpose(1, 2)).transpose(1, 2)\\n\\n        query = attn.to_q(hidden_states)\\n\\n        if encoder_hidden_states is None:\\n            encoder_hidden_states = hidden_states\\n        else:\\n            end_pos = encoder_hidden_states.shape[1] - self.num_tokens\\n            encoder_hidden_states = encoder_hidden_states[:, :end_pos]  # only use text\\n            if attn.norm_cross:\\n                encoder_hidden_states = attn.norm_encoder_hidden_states(encoder_hidden_states)\\n\\n        key = attn.to_k(encoder_hidden_states)\\n        value = attn.to_v(encoder_hidden_states)\\n\\n        inner_dim = key.shape[-1]\\n        head_dim = inner_dim // attn.heads\\n\\n        query = query.view(batch_size, -1, attn.heads, head_dim).transpose(1, 2)\\n\\n        key = key.view(batch_size, -1, attn.heads, head_dim).transpose(1, 2)\\n        value = value.view(batch_size, -1, attn.heads, head_dim).transpose(1, 2)\\n\\n        # the output of sdp = (batch, num_heads, seq_len, head_dim)\\n        # TODO: add support for attn.scale when we move to Torch 2.1\\n        hidden_states = F.scaled_dot_product_attention(\\n            query, key, value, attn_mask=attention_mask, dropout_p=0.0, is_causal=False\\n        )\\n\\n        hidden_states = hidden_states.transpose(1, 2).reshape(batch_size, -1, attn.heads * head_dim)\\n        hidden_states = hidden_states.to(query.dtype)\\n\\n        # linear proj\\n        hidden_states = attn.to_out[0](hidden_states)\\n        # dropout\\n        hidden_states = attn.to_out[1](hidden_states)\\n\\n        if input_ndim == 4:\\n            hidden_states = hidden_states.transpose(-1, -2).reshape(batch_size, channel, height, width)\\n\\n        if attn.residual_connection:\\n            hidden_states = hidden_states + residual\\n\\n        hidden_states = hidden_states / attn.rescale_output_factor\\n\\n        return hidden_states\\n\\n\\nimport torch\\nimport torch.nn.functional as F\\nimport numpy as np\\nfrom PIL import Image\\n\\nattn_maps = {}\\ndef hook_fn(name):\\n    def forward_hook(module, input, output):\\n        if hasattr(module.processor, \\\"attn_map\\\"):\\n            attn_maps[name] = module.processor.attn_map\\n            del module.processor.attn_map\\n\\n    return forward_hook\\n\\ndef register_cross_attention_hook(unet):\\n    for name, module in unet.named_modules():\\n        if name.split('.')[-1].startswith('attn2'):\\n            module.register_forward_hook(hook_fn(name))\\n\\n    return unet\\n\\ndef upscale(attn_map, target_size):\\n    attn_map = torch.mean(attn_map, dim=0)\\n    attn_map = attn_map.permute(1,0)\\n    temp_size = None\\n\\n    for i in range(0,5):\\n        scale = 2 ** i\\n        if ( target_size[0] // scale ) * ( target_size[1] // scale) == attn_map.shape[1]*64:\\n            temp_size = (target_size[0]//(scale*8), target_size[1]//(scale*8))\\n            break\\n\\n    assert temp_size is not None, \\\"temp_size cannot is None\\\"\\n\\n    attn_map = attn_map.view(attn_map.shape[0], *temp_size)\\n\\n    attn_map = F.interpolate(\\n        attn_map.unsqueeze(0).to(dtype=torch.float32),\\n        size=target_size,\\n        mode='bilinear',\\n        align_corners=False\\n    )[0]\\n\\n    attn_map = torch.softmax(attn_map, dim=0)\\n    return attn_map\\ndef get_net_attn_map(image_size, batch_size=2, instance_or_negative=False, detach=True):\\n\\n    idx = 0 if instance_or_negative else 1\\n    net_attn_maps = []\\n\\n    for name, attn_map in attn_maps.items():\\n        attn_map = attn_map.cpu() if detach else attn_map\\n        attn_map = torch.chunk(attn_map, batch_size)[idx].squeeze()\\n        attn_map = upscale(attn_map, image_size) \\n        net_attn_maps.append(attn_map) \\n\\n    net_attn_maps = torch.mean(torch.stack(net_attn_maps,dim=0),dim=0)\\n\\n    return net_attn_maps\\n\\ndef attnmaps2images(net_attn_maps):\\n\\n    #total_attn_scores = 0\\n    images = []\\n\\n    for attn_map in net_attn_maps:\\n        attn_map = attn_map.cpu().numpy()\\n        #total_attn_scores += attn_map.mean().item()\\n\\n        normalized_attn_map = (attn_map - np.min(attn_map)) / (np.max(attn_map) - np.min(attn_map)) * 255\\n        normalized_attn_map = normalized_attn_map.astype(np.uint8)\\n        #print(\\\"norm: \\\", normalized_attn_map.shape)\\n        image = Image.fromarray(normalized_attn_map)\\n\\n        #image = fix_save_attn_map(attn_map)\\n        images.append(image)\\n\\n    #print(total_attn_scores)\\n    return images\\ndef is_torch2_available():\\n    return hasattr(F, \\\"scaled_dot_product_attention\\\")\\n\\ndef get_generator(seed, device):\\n\\n    if seed is not None:\\n        if isinstance(seed, list):\\n            generator = [torch.Generator(device).manual_seed(seed_item) for seed_item in seed]\\n        else:\\n            generator = torch.Generator(device).manual_seed(seed)\\n    else:\\n        generator = None\\n\\n    return generator\\n\\nimport os\\nfrom typing import List\\n\\nimport torch\\nfrom diffusers import StableDiffusionPipeline\\nfrom diffusers.pipelines.controlnet import MultiControlNetModel\\nfrom PIL import Image\\nfrom safetensors import safe_open\\nfrom transformers import CLIPImageProcessor, CLIPVisionModelWithProjection\\n\\nfrom .utils import is_torch2_available, get_generator\\n\\nif is_torch2_available():\\n    from .attention_processor import (\\n        AttnProcessor2_0 as AttnProcessor,\\n    )\\n    from .attention_processor import (\\n        CNAttnProcessor2_0 as CNAttnProcessor,\\n    )\\n    from .attention_processor import (\\n        IPAttnProcessor2_0 as IPAttnProcessor,\\n    )\\nelse:\\n    from .attention_processor import AttnProcessor, CNAttnProcessor, IPAttnProcessor\\nfrom .resampler import Resampler\\n\\n\\nclass ImageProjModel(torch.nn.Module):\\n    \\\"\\\"\\\"Projection Model\\\"\\\"\\\"\\n\\n    def __init__(self, cross_attention_dim=1024, clip_embeddings_dim=1024, clip_extra_context_tokens=4):\\n        super().__init__()\\n\\n        self.generator = None\\n        self.cross_attention_dim = cross_attention_dim\\n        self.clip_extra_context_tokens = clip_extra_context_tokens\\n        self.proj = torch.nn.Linear(clip_embeddings_dim, self.clip_extra_context_tokens * cross_attention_dim)\\n        self.norm = torch.nn.LayerNorm(cross_attention_dim)\\n\\n    def forward(self, image_embeds):\\n        embeds = image_embeds\\n        clip_extra_context_tokens = self.proj(embeds).reshape(\\n            -1, self.clip_extra_context_tokens, self.cross_attention_dim\\n        )\\n        clip_extra_context_tokens = self.norm(clip_extra_context_tokens)\\n        return clip_extra_context_tokens\\n\\n\\nclass MLPProjModel(torch.nn.Module):\\n    \\\"\\\"\\\"SD model with image prompt\\\"\\\"\\\"\\n    def __init__(self, cross_attention_dim=1024, clip_embeddings_dim=1024):\\n        super().__init__()\\n        \\n        self.proj = torch.nn.Sequential(\\n            torch.nn.Linear(clip_embeddings_dim, clip_embeddings_dim),\\n            torch.nn.GELU(),\\n            torch.nn.Linear(clip_embeddings_dim, cross_attention_dim),\\n            torch.nn.LayerNorm(cross_attention_dim)\\n        )\\n        \\n    def forward(self, image_embeds):\\n        clip_extra_context_tokens = self.proj(image_embeds)\\n        return clip_extra_context_tokens\\n\\n\\nclass IPAdapter:\\n    def __init__(self, sd_pipe, image_encoder_path, ip_ckpt, device, num_tokens=4, target_blocks=[\\\"block\\\"]):\\n        self.device = device\\n        self.image_encoder_path = image_encoder_path\\n        self.ip_ckpt = ip_ckpt\\n        self.num_tokens = num_tokens\\n        self.target_blocks = target_blocks\\n\\n        self.pipe = sd_pipe.to(self.device)\\n        self.set_ip_adapter()\\n\\n        # load image encoder\\n        self.image_encoder = CLIPVisionModelWithProjection.from_pretrained(self.image_encoder_path).to(\\n            self.device, dtype=torch.float16\\n        )\\n        self.clip_image_processor = CLIPImageProcessor()\\n        # image proj model\\n        self.image_proj_model = self.init_proj()\\n\\n        self.load_ip_adapter()\\n\\n    def init_proj(self):\\n        image_proj_model = ImageProjModel(\\n            cross_attention_dim=self.pipe.unet.config.cross_attention_dim,\\n            clip_embeddings_dim=self.image_encoder.config.projection_dim,\\n            clip_extra_context_tokens=self.num_tokens,\\n        ).to(self.device, dtype=torch.float16)\\n        return image_proj_model\\n\\n    def set_ip_adapter(self):\\n        unet = self.pipe.unet\\n        attn_procs = {}\\n        for name in unet.attn_processors.keys():\\n            cross_attention_dim = None if name.endswith(\\\"attn1.processor\\\") else unet.config.cross_attention_dim\\n            if name.startswith(\\\"mid_block\\\"):\\n                hidden_size = unet.config.block_out_channels[-1]\\n            elif name.startswith(\\\"up_blocks\\\"):\\n                block_id = int(name[len(\\\"up_blocks.\\\")])\\n                hidden_size = list(reversed(unet.config.block_out_channels))[block_id]\\n            elif name.startswith(\\\"down_blocks\\\"):\\n                block_id = int(name[len(\\\"down_blocks.\\\")])\\n                hidden_size = unet.config.block_out_channels[block_id]\\n            if cross_attention_dim is None:\\n                attn_procs[name] = AttnProcessor()\\n            else:\\n                selected = False\\n                for block_name in self.target_blocks:\\n                    if block_name in name:\\n                        selected = True\\n                        break\\n                if selected:\\n                    attn_procs[name] = IPAttnProcessor(\\n                        hidden_size=hidden_size,\\n                        cross_attention_dim=cross_attention_dim,\\n                        scale=1.0,\\n                        num_tokens=self.num_tokens,\\n                    ).to(self.device, dtype=torch.float16)\\n                else:\\n                    attn_procs[name] = IPAttnProcessor(\\n                        hidden_size=hidden_size,\\n                        cross_attention_dim=cross_attention_dim,\\n                        scale=1.0,\\n                        num_tokens=self.num_tokens,\\n                        skip=True\\n                    ).to(self.device, dtype=torch.float16)\\n        unet.set_attn_processor(attn_procs)\\n        if hasattr(self.pipe, \\\"controlnet\\\"):\\n            if isinstance(self.pipe.controlnet, MultiControlNetModel):\\n                for controlnet in self.pipe.controlnet.nets:\\n                    controlnet.set_attn_processor(CNAttnProcessor(num_tokens=self.num_tokens))\\n            else:\\n                self.pipe.controlnet.set_attn_processor(CNAttnProcessor(num_tokens=self.num_tokens))\\n\\n    def load_ip_adapter(self):\\n        if os.path.splitext(self.ip_ckpt)[-1] == \\\".safetensors\\\":\\n            state_dict = {\\\"image_proj\\\": {}, \\\"ip_adapter\\\": {}}\\n            with safe_open(self.ip_ckpt, framework=\\\"pt\\\", device=\\\"cpu\\\") as f:\\n                for key in f.keys():\\n                    if key.startswith(\\\"image_proj.\\\"):\\n                        state_dict[\\\"image_proj\\\"][key.replace(\\\"image_proj.\\\", \\\"\\\")] = f.get_tensor(key)\\n                    elif key.startswith(\\\"ip_adapter.\\\"):\\n                        state_dict[\\\"ip_adapter\\\"][key.replace(\\\"ip_adapter.\\\", \\\"\\\")] = f.get_tensor(key)\\n        else:\\n            state_dict = torch.load(self.ip_ckpt,weights_only=True, map_location=\\\"cuda\\\" if torch.cuda.is_available() else \\\"cpu\\\")\\n        self.image_proj_model.load_state_dict(state_dict[\\\"image_proj\\\"])\\n        ip_layers = torch.nn.ModuleList(self.pipe.unet.attn_processors.values())\\n        ip_layers.load_state_dict(state_dict[\\\"ip_adapter\\\"], strict=False)\\n\\n    @torch.inference_mode()\\n    def get_image_embeds(self, pil_image=None, clip_image_embeds=None, content_prompt_embeds=None):\\n        if pil_image is not None:\\n            if isinstance(pil_image, Image.Image):\\n                pil_image = [pil_image]\\n            clip_image = self.clip_image_processor(images=pil_image, return_tensors=\\\"pt\\\").pixel_values\\n            clip_image_embeds = self.image_encoder(clip_image.to(self.device, dtype=torch.float16)).image_embeds\\n        else:\\n            clip_image_embeds = clip_image_embeds.to(self.device, dtype=torch.float16)\\n        \\n        if content_prompt_embeds is not None:\\n            clip_image_embeds = clip_image_embeds - content_prompt_embeds\\n\\n        image_prompt_embeds = self.image_proj_model(clip_image_embeds)\\n        uncond_image_prompt_embeds = self.image_proj_model(torch.zeros_like(clip_image_embeds))\\n        return image_prompt_embeds, uncond_image_prompt_embeds\\n\\n    def set_scale(self, scale):\\n        for attn_processor in self.pipe.unet.attn_processors.values():\\n            if isinstance(attn_processor, IPAttnProcessor):\\n                attn_processor.scale = scale\\n\\n    def generate(\\n        self,\\n        pil_image=None,\\n        clip_image_embeds=None,\\n        prompt=None,\\n        negative_prompt=None,\\n        scale=1.0,\\n        num_samples=4,\\n        seed=None,\\n        guidance_scale=7.5,\\n        num_inference_steps=30,\\n        neg_content_emb=None,\\n        **kwargs,\\n    ):\\n        self.set_scale(scale)\\n\\n        if pil_image is not None:\\n            num_prompts = 1 if isinstance(pil_image, Image.Image) else len(pil_image)\\n        else:\\n            num_prompts = clip_image_embeds.size(0)\\n\\n        if prompt is None:\\n            prompt = \\\"best quality, high quality\\\"\\n        if negative_prompt is None:\\n            negative_prompt = \\\"monochrome, lowres, bad anatomy, worst quality, low quality\\\"\\n\\n        if not isinstance(prompt, List):\\n            prompt = [prompt] * num_prompts\\n        if not isinstance(negative_prompt, List):\\n            negative_prompt = [negative_prompt] * num_prompts\\n\\n        image_prompt_embeds, uncond_image_prompt_embeds = self.get_image_embeds(\\n            pil_image=pil_image, clip_image_embeds=clip_image_embeds, content_prompt_embeds=neg_content_emb\\n        )\\n        bs_embed, seq_len, _ = image_prompt_embeds.shape\\n        image_prompt_embeds = image_prompt_embeds.repeat(1, num_samples, 1)\\n        image_prompt_embeds = image_prompt_embeds.view(bs_embed * num_samples, seq_len, -1)\\n        uncond_image_prompt_embeds = uncond_image_prompt_embeds.repeat(1, num_samples, 1)\\n        uncond_image_prompt_embeds = uncond_image_prompt_embeds.view(bs_embed * num_samples, seq_len, -1)\\n\\n        with torch.inference_mode():\\n            prompt_embeds_, negative_prompt_embeds_ = self.pipe.encode_prompt(\\n                prompt,\\n                device=self.device,\\n                num_images_per_prompt=num_samples,\\n                do_classifier_free_guidance=True,\\n                negative_prompt=negative_prompt,\\n            )\\n            prompt_embeds = torch.cat([prompt_embeds_, image_prompt_embeds], dim=1)\\n            negative_prompt_embeds = torch.cat([negative_prompt_embeds_, uncond_image_prompt_embeds], dim=1)\\n\\n        generator = get_generator(seed, self.device)\\n\\n        images = self.pipe(\\n            prompt_embeds=prompt_embeds,\\n            negative_prompt_embeds=negative_prompt_embeds,\\n            guidance_scale=guidance_scale,\\n            num_inference_steps=num_inference_steps,\\n            generator=generator,\\n            **kwargs,\\n        ).images\\n\\n        return images\\n\\n\\nclass IPAdapterXL(IPAdapter):\\n    \\\"\\\"\\\"SDXL\\\"\\\"\\\"\\n\\n    def generate(\\n        self,\\n        pil_image,\\n        prompt=None,\\n        negative_prompt=None,\\n        scale=1.0,\\n        num_samples=4,\\n        seed=None,\\n        num_inference_steps=30,\\n        neg_content_emb=None,\\n        neg_content_prompt=None,\\n        neg_content_scale=1.0,\\n        **kwargs,\\n    ):\\n        self.set_scale(scale)\\n\\n        num_prompts = 1 if isinstance(pil_image, Image.Image) else len(pil_image)\\n\\n        if prompt is None:\\n            prompt = \\\"best quality, high quality\\\"\\n        if negative_prompt is None:\\n            negative_prompt = \\\"monochrome, lowres, bad anatomy, worst quality, low quality\\\"\\n\\n        if not isinstance(prompt, List):\\n            prompt = [prompt] * num_prompts\\n        if not isinstance(negative_prompt, List):\\n            negative_prompt = [negative_prompt] * num_prompts\\n        \\n        if neg_content_emb is None:\\n            if neg_content_prompt is not None:\\n                with torch.inference_mode():\\n                    (\\n                        prompt_embeds_, # torch.Size([1, 77, 2048])\\n                        negative_prompt_embeds_,\\n                        pooled_prompt_embeds_, # torch.Size([1, 1280])\\n                        negative_pooled_prompt_embeds_,\\n                    ) = self.pipe.encode_prompt(\\n                        neg_content_prompt,\\n                        num_images_per_prompt=num_samples,\\n                        do_classifier_free_guidance=True,\\n                        negative_prompt=negative_prompt,\\n                    )\\n                    pooled_prompt_embeds_ *= neg_content_scale\\n            else:\\n                pooled_prompt_embeds_ = neg_content_emb\\n        else:\\n            pooled_prompt_embeds_ = None\\n\\n        image_prompt_embeds, uncond_image_prompt_embeds = self.get_image_embeds(pil_image, content_prompt_embeds=pooled_prompt_embeds_)\\n        bs_embed, seq_len, _ = image_prompt_embeds.shape\\n        image_prompt_embeds = image_prompt_embeds.repeat(1, num_samples, 1)\\n        image_prompt_embeds = image_prompt_embeds.view(bs_embed * num_samples, seq_len, -1)\\n        uncond_image_prompt_embeds = uncond_image_prompt_embeds.repeat(1, num_samples, 1)\\n        uncond_image_prompt_embeds = uncond_image_prompt_embeds.view(bs_embed * num_samples, seq_len, -1)\\n\\n        with torch.inference_mode():\\n            (\\n                prompt_embeds,\\n                negative_prompt_embeds,\\n                pooled_prompt_embeds,\\n                negative_pooled_prompt_embeds,\\n            ) = self.pipe.encode_prompt(\\n                prompt,\\n                num_images_per_prompt=num_samples,\\n                do_classifier_free_guidance=True,\\n                negative_prompt=negative_prompt,\\n            )\\n            prompt_embeds = torch.cat([prompt_embeds, image_prompt_embeds], dim=1)\\n            negative_prompt_embeds = torch.cat([negative_prompt_embeds, uncond_image_prompt_embeds], dim=1)\\n\\n        self.generator = get_generator(seed, self.device)\\n        \\n        images = self.pipe(\\n            prompt_embeds=prompt_embeds,\\n            negative_prompt_embeds=negative_prompt_embeds,\\n            pooled_prompt_embeds=pooled_prompt_embeds,\\n            negative_pooled_prompt_embeds=negative_pooled_prompt_embeds,\\n            num_inference_steps=num_inference_steps,\\n            generator=self.generator,\\n            **kwargs,\\n        ).images\\n\\n        return images\\n\\n\\nclass IPAdapterPlus(IPAdapter):\\n    \\\"\\\"\\\"IP-Adapter with fine-grained features\\\"\\\"\\\"\\n\\n    def init_proj(self):\\n        image_proj_model = Resampler(\\n            dim=self.pipe.unet.config.cross_attention_dim,\\n            depth=4,\\n            dim_head=64,\\n            heads=12,\\n            num_queries=self.num_tokens,\\n            embedding_dim=self.image_encoder.config.hidden_size,\\n            output_dim=self.pipe.unet.config.cross_attention_dim,\\n            ff_mult=4,\\n        ).to(self.device, dtype=torch.float16)\\n        return image_proj_model\\n\\n    @torch.inference_mode()\\n    def get_image_embeds(self, pil_image=None, clip_image_embeds=None):\\n        if isinstance(pil_image, Image.Image):\\n            pil_image = [pil_image]\\n        clip_image = self.clip_image_processor(images=pil_image, return_tensors=\\\"pt\\\").pixel_values\\n        clip_image = clip_image.to(self.device, dtype=torch.float16)\\n        clip_image_embeds = self.image_encoder(clip_image, output_hidden_states=True).hidden_states[-2]\\n        image_prompt_embeds = self.image_proj_model(clip_image_embeds)\\n        uncond_clip_image_embeds = self.image_encoder(\\n            torch.zeros_like(clip_image), output_hidden_states=True\\n        ).hidden_states[-2]\\n        uncond_image_prompt_embeds = self.image_proj_model(uncond_clip_image_embeds)\\n        return image_prompt_embeds, uncond_image_prompt_embeds\\n\\n\\nclass IPAdapterFull(IPAdapterPlus):\\n    \\\"\\\"\\\"IP-Adapter with full features\\\"\\\"\\\"\\n\\n    def init_proj(self):\\n        image_proj_model = MLPProjModel(\\n            cross_attention_dim=self.pipe.unet.config.cross_attention_dim,\\n            clip_embeddings_dim=self.image_encoder.config.hidden_size,\\n        ).to(self.device, dtype=torch.float16)\\n        return image_proj_model\\n\\n\\nclass IPAdapterPlusXL(IPAdapter):\\n    \\\"\\\"\\\"SDXL\\\"\\\"\\\"\\n\\n    def init_proj(self):\\n        image_proj_model = Resampler(\\n            dim=1280,\\n            depth=4,\\n            dim_head=64,\\n            heads=20,\\n            num_queries=self.num_tokens,\\n            embedding_dim=self.image_encoder.config.hidden_size,\\n            output_dim=self.pipe.unet.config.cross_attention_dim,\\n            ff_mult=4,\\n        ).to(self.device, dtype=torch.float16)\\n        return image_proj_model\\n\\n    @torch.inference_mode()\\n    def get_image_embeds(self, pil_image):\\n        if isinstance(pil_image, Image.Image):\\n            pil_image = [pil_image]\\n        clip_image = self.clip_image_processor(images=pil_image, return_tensors=\\\"pt\\\").pixel_values\\n        clip_image = clip_image.to(self.device, dtype=torch.float16)\\n        clip_image_embeds = self.image_encoder(clip_image, output_hidden_states=True).hidden_states[-2]\\n        image_prompt_embeds = self.image_proj_model(clip_image_embeds)\\n        uncond_clip_image_embeds = self.image_encoder(\\n            torch.zeros_like(clip_image), output_hidden_states=True\\n        ).hidden_states[-2]\\n        uncond_image_prompt_embeds = self.image_proj_model(uncond_clip_image_embeds)\\n        return image_prompt_embeds, uncond_image_prompt_embeds\\n\\n    def generate(\\n        self,\\n        pil_image,\\n        prompt=None,\\n        negative_prompt=None,\\n        scale=1.0,\\n        num_samples=4,\\n        seed=None,\\n        num_inference_steps=30,\\n        **kwargs,\\n    ):\\n        self.set_scale(scale)\\n\\n        num_prompts = 1 if isinstance(pil_image, Image.Image) else len(pil_image)\\n\\n        if prompt is None:\\n            prompt = \\\"best quality, high quality\\\"\\n        if negative_prompt is None:\\n            negative_prompt = \\\"monochrome, lowres, bad anatomy, worst quality, low quality\\\"\\n\\n        if not isinstance(prompt, List):\\n            prompt = [prompt] * num_prompts\\n        if not isinstance(negative_prompt, List):\\n            negative_prompt = [negative_prompt] * num_prompts\\n\\n        image_prompt_embeds, uncond_image_prompt_embeds = self.get_image_embeds(pil_image)\\n        bs_embed, seq_len, _ = image_prompt_embeds.shape\\n        image_prompt_embeds = image_prompt_embeds.repeat(1, num_samples, 1)\\n        image_prompt_embeds = image_prompt_embeds.view(bs_embed * num_samples, seq_len, -1)\\n        uncond_image_prompt_embeds = uncond_image_prompt_embeds.repeat(1, num_samples, 1)\\n        uncond_image_prompt_embeds = uncond_image_prompt_embeds.view(bs_embed * num_samples, seq_len, -1)\\n\\n        with torch.inference_mode():\\n            (\\n                prompt_embeds,\\n                negative_prompt_embeds,\\n                pooled_prompt_embeds,\\n                negative_pooled_prompt_embeds,\\n            ) = self.pipe.encode_prompt(\\n                prompt,\\n                num_images_per_prompt=num_samples,\\n                do_classifier_free_guidance=True,\\n                negative_prompt=negative_prompt,\\n            )\\n            prompt_embeds = torch.cat([prompt_embeds, image_prompt_embeds], dim=1)\\n            negative_prompt_embeds = torch.cat([negative_prompt_embeds, uncond_image_prompt_embeds], dim=1)\\n\\n        generator = get_generator(seed, self.device)\\n\\n        images = self.pipe(\\n            prompt_embeds=prompt_embeds,\\n            negative_prompt_embeds=negative_prompt_embeds,\\n            pooled_prompt_embeds=pooled_prompt_embeds,\\n            negative_pooled_prompt_embeds=negative_pooled_prompt_embeds,\\n            num_inference_steps=num_inference_steps,\\n            generator=generator,\\n            **kwargs,\\n        ).images\\n\\n        return images\\n\\n\\n# modified from https://github.com/mlfoundations/open_flamingo/blob/main/open_flamingo/src/helpers.py\\n# and https://github.com/lucidrains/imagen-pytorch/blob/main/imagen_pytorch/imagen_pytorch.py\\n\\nimport math\\n\\nimport torch\\nimport torch.nn as nn\\nfrom einops import rearrange\\nfrom einops.layers.torch import Rearrange\\n\\n\\n# FFN\\ndef FeedForward(dim, mult=4):\\n    inner_dim = int(dim * mult)\\n    return nn.Sequential(\\n        nn.LayerNorm(dim),\\n        nn.Linear(dim, inner_dim, bias=False),\\n        nn.GELU(),\\n        nn.Linear(inner_dim, dim, bias=False),\\n    )\\n\\n\\ndef reshape_tensor(x, heads):\\n    bs, length, width = x.shape\\n    # (bs, length, width) --> (bs, length, n_heads, dim_per_head)\\n    x = x.view(bs, length, heads, -1)\\n    # (bs, length, n_heads, dim_per_head) --> (bs, n_heads, length, dim_per_head)\\n    x = x.transpose(1, 2)\\n    # (bs, n_heads, length, dim_per_head) --> (bs*n_heads, length, dim_per_head)\\n    x = x.reshape(bs, heads, length, -1)\\n    return x\\n\\n\\nclass PerceiverAttention(nn.Module):\\n    def __init__(self, *, dim, dim_head=64, heads=8):\\n        super().__init__()\\n        self.scale = dim_head**-0.5\\n        self.dim_head = dim_head\\n        self.heads = heads\\n        inner_dim = dim_head * heads\\n\\n        self.norm1 = nn.LayerNorm(dim)\\n        self.norm2 = nn.LayerNorm(dim)\\n\\n        self.to_q = nn.Linear(dim, inner_dim, bias=False)\\n        self.to_kv = nn.Linear(dim, inner_dim * 2, bias=False)\\n        self.to_out = nn.Linear(inner_dim, dim, bias=False)\\n\\n    def forward(self, x, latents):\\n        \\\"\\\"\\\"\\n        Args:\\n            x (torch.Tensor): image features\\n                shape (b, n1, D)\\n            latent (torch.Tensor): latent features\\n                shape (b, n2, D)\\n        \\\"\\\"\\\"\\n        x = self.norm1(x)\\n        latents = self.norm2(latents)\\n\\n        b, l, _ = latents.shape\\n\\n        q = self.to_q(latents)\\n        kv_input = torch.cat((x, latents), dim=-2)\\n        k, v = self.to_kv(kv_input).chunk(2, dim=-1)\\n\\n        q = reshape_tensor(q, self.heads)\\n        k = reshape_tensor(k, self.heads)\\n        v = reshape_tensor(v, self.heads)\\n\\n        # attention\\n        scale = 1 / math.sqrt(math.sqrt(self.dim_head))\\n        weight = (q * scale) @ (k * scale).transpose(-2, -1)  # More stable with f16 than dividing afterwards\\n        weight = torch.softmax(weight.float(), dim=-1).type(weight.dtype)\\n        out = weight @ v\\n\\n        out = out.permute(0, 2, 1, 3).reshape(b, l, -1)\\n\\n        return self.to_out(out)\\n\\n\\nclass Resampler(nn.Module):\\n    def __init__(\\n        self,\\n        dim=1024,\\n        depth=8,\\n        dim_head=64,\\n        heads=16,\\n        num_queries=8,\\n        embedding_dim=768,\\n        output_dim=1024,\\n        ff_mult=4,\\n        max_seq_len: int = 257,  # CLIP tokens + CLS token\\n        apply_pos_emb: bool = False,\\n        num_latents_mean_pooled: int = 0,  # number of latents derived from mean pooled representation of the sequence\\n    ):\\n        super().__init__()\\n        self.pos_emb = nn.Embedding(max_seq_len, embedding_dim) if apply_pos_emb else None\\n\\n        self.latents = nn.Parameter(torch.randn(1, num_queries, dim) / dim**0.5)\\n\\n        self.proj_in = nn.Linear(embedding_dim, dim)\\n\\n        self.proj_out = nn.Linear(dim, output_dim)\\n        self.norm_out = nn.LayerNorm(output_dim)\\n\\n        self.to_latents_from_mean_pooled_seq = (\\n            nn.Sequential(\\n                nn.LayerNorm(dim),\\n                nn.Linear(dim, dim * num_latents_mean_pooled),\\n                Rearrange(\\\"b (n d) -> b n d\\\", n=num_latents_mean_pooled),\\n            )\\n            if num_latents_mean_pooled > 0\\n            else None\\n        )\\n\\n        self.layers = nn.ModuleList([])\\n        for _ in range(depth):\\n            self.layers.append(\\n                nn.ModuleList(\\n                    [\\n                        PerceiverAttention(dim=dim, dim_head=dim_head, heads=heads),\\n                        FeedForward(dim=dim, mult=ff_mult),\\n                    ]\\n                )\\n            )\\n\\n    def forward(self, x):\\n        if self.pos_emb is not None:\\n            n, device = x.shape[1], x.device\\n            pos_emb = self.pos_emb(torch.arange(n, device=device))\\n            x = x + pos_emb\\n\\n        latents = self.latents.repeat(x.size(0), 1, 1)\\n\\n        x = self.proj_in(x)\\n\\n        if self.to_latents_from_mean_pooled_seq:\\n            meanpooled_seq = masked_mean(x, dim=1, mask=torch.ones(x.shape[:2], device=x.device, dtype=torch.bool))\\n            meanpooled_latents = self.to_latents_from_mean_pooled_seq(meanpooled_seq)\\n            latents = torch.cat((meanpooled_latents, latents), dim=-2)\\n\\n        for attn, ff in self.layers:\\n            latents = attn(x, latents) + latents\\n            latents = ff(latents) + latents\\n\\n        latents = self.proj_out(latents)\\n        return self.norm_out(latents)\\n\\n\\ndef masked_mean(t, *, dim, mask=None):\\n    if mask is None:\\n        return t.mean(dim=dim)\\n\\n    denom = mask.sum(dim=dim, keepdim=True)\\n    mask = rearrange(mask, \\\"b n -> b n 1\\\")\\n    masked_t = t.masked_fill(~mask, 0.0)\\n\\n    return masked_t.sum(dim=dim) / denom.clamp(min=1e-5)\\n\\n\\nfrom .ip_adapter import IPAdapter, IPAdapterPlus, IPAdapterPlusXL, IPAdapterXL, IPAdapterFull\\n\\n__all__ = [\\n    \\\"IPAdapter\\\",\\n    \\\"IPAdapterPlus\\\",\\n    \\\"IPAdapterPlusXL\\\",\\n    \\\"IPAdapterXL\\\",\\n    \\\"IPAdapterFull\\\",\\n]\\n\\n\\nimport os\\nimport random\\nimport sys\\nfrom typing import Sequence, Mapping, Any, Union, List\\nimport torch\\nfrom fastapi import APIRouter, UploadFile, File\\nfrom fastapi.responses import FileResponse\\n\\nimport os\\nfrom routers import ImageResponse, SAVE_PATH, OUTPUT_PATH\\nfrom routers import get_value_at_index\\n\\n\\n\\n\\nfrom nodes import SaveImage, LoadImage, NODE_CLASS_MAPPINGS\\n\\nrouter = APIRouter(\\n    prefix=\\\"/product\\\",\\n    tags=[\\\"dress_try\\\"],\\n    dependencies=[],\\n    responses={404: {\\\"description\\\": \\\"Not found\\\"}},\\n)\\n\\n@router.post(\\\"/dress_try\\\")\\n# NOTE:ch_type: \\\"Half body\\\" or \\\"Full body\\\"\\n# NOTE:ch_category: \\\"Upper body\\\" or \\\"Lower body\\\" or \\\"Full body\\\"\\nasync def dress_try(ch_type: str=\\\"Half body\\\", ch_category: str=\\\"Upper body\\\", modelimage: UploadFile = File(...), clothimage: UploadFile = File(...)):\\n    if ch_type not in [\\\"Half body\\\", \\\"Full body\\\"]:\\n        result = ImageResponse()\\n        result.data = []\\n        result.code = 400\\n        result.message = \\\"Invalid ch_type!\\\"\\n        return result\\n\\n    if ch_category not in [\\\"Upper body\\\", \\\"Lower body\\\", \\\"Full body\\\"]:\\n        result = ImageResponse()\\n        result.data = []\\n        result.code = 400\\n        result.message = \\\"Invalid ch_category!\\\"\\n        return result\\n    \\n    if ch_type == \\\"Half body\\\" and ch_category != \\\"Upper body\\\":\\n        result = ImageResponse()\\n        result.data = []\\n        result.code = 400\\n        result.message = \\\"Half body (hd) model type can only be used with upperbody category!\\\"\\n        return result\\n    \\n    # 将上传的图片保存到本地\\n    model_image_path = os.path.join(SAVE_PATH, modelimage.filename)\\n    with open(model_image_path, \\\"wb\\\") as buffer:\\n        buffer.write(modelimage.file.read())\\n    \\n    cloth_image_path = os.path.join(SAVE_PATH, clothimage.filename)\\n    with open(cloth_image_path, \\\"wb\\\") as buffer:\\n        buffer.write(clothimage.file.read())\\n    \\n    with torch.inference_mode():\\n        loadimage = LoadImage()\\n        loadimage_1 = loadimage.load_image(image=modelimage.filename)\\n\\n        loadimage_4 = loadimage.load_image(image=clothimage.filename)\\n\\n        loadootdpipeline = NODE_CLASS_MAPPINGS[\\\"LoadOOTDPipeline\\\"]()\\n        loadootdpipeline_11 = loadootdpipeline.load(\\n            type=ch_type, path=\\\"/root/ComfyUI/models/OOTDiffusion\\\"\\n        )\\n\\n        ootdgenerate = NODE_CLASS_MAPPINGS[\\\"OOTDGenerate\\\"]()\\n        saveimage = SaveImage()\\n\\n        try:\\n            img_paths = []\\n            for q in range(1):\\n                ootdgenerate_3 = ootdgenerate.generate(\\n                    seed=random.randint(1, 2**64),\\n                    steps=20,\\n                    cfg=2.0300000000000002,\\n                    category=ch_category,\\n                    pipe=get_value_at_index(loadootdpipeline_11, 0),\\n                    cloth_image=get_value_at_index(loadimage_4, 0),\\n                    model_image=get_value_at_index(loadimage_1, 0),\\n                )\\n\\n                saveimage_12 = saveimage.save_images(\\n                    filename_prefix=\\\"ComfyUI\\\", images=get_value_at_index(ootdgenerate_3, 0)\\n                )\\n                img_paths.append(saveimage_12['ui']['images'][0]['filename'])\\n                \\n                # 将生成的图片路径转换为 FileResponse 列表\\n                file_responses: List[FileResponse] = []\\n\\n                for path in img_paths:\\n                    save_path = os.path.join(OUTPUT_PATH, path)\\n                    file_responses.append(FileResponse(save_path))\\n                return file_responses[0]\\n            \\n        except:\\n            result = ImageResponse()\\n            result.data = []\\n            result.code = 500\\n            result.message = \\\"Model inference Error!\\\"\\n            return result\\n\\n\\n\\nimport os\\nimport shutil\\nimport random\\nimport asyncio\\nfrom typing import  List, AsyncGenerator\\nimport torch\\n\\nfrom fastapi import APIRouter, UploadFile, File\\nfrom fastapi.responses import FileResponse, StreamingResponse\\n\\nimport ast\\nimport uuid\\n\\nimport models.fileModel as fileModel\\nfrom routers import ImageResponse, SAVE_PATH, OUTPUT_PATH\\nfrom routers import  get_value_at_index\\nfrom routers import image_style\\n\\nfrom nodes import (\\n    NODE_CLASS_MAPPINGS,\\n    EmptyLatentImage,\\n    UNETLoader,\\n    SaveImage,\\n    VAEDecode,\\n    CLIPTextEncode,\\n    DualCLIPLoader,\\n    VAELoader,\\n)\\n\\nwith torch.no_grad():\\n    dualcliploader = DualCLIPLoader()\\n    dualcliploader_61 = dualcliploader.load_clip(\\n        clip_name1=\\\"t5xxl_fp8_e4m3fn.safetensors\\\",\\n        clip_name2=\\\"clip_l.safetensors\\\",\\n        type=\\\"flux\\\",\\n    )\\n\\n    vaeloader = VAELoader()\\n    vaeloader_10 = vaeloader.load_vae(vae_name=\\\"ae.safetensors\\\")\\n\\n    unetloader = UNETLoader()\\n    unetloader_12 = unetloader.load_unet(\\n        unet_name=\\\"flux1-dev.safetensors\\\", weight_dtype=\\\"default\\\"\\n    )\\n\\n    ksamplerselect = NODE_CLASS_MAPPINGS[\\\"KSamplerSelect\\\"]()\\n    ksamplerselect_16 = ksamplerselect.get_sampler(sampler_name=\\\"euler\\\")\\n\\n    randomnoise = NODE_CLASS_MAPPINGS[\\\"RandomNoise\\\"]()\\n    randomnoise_25 = randomnoise.get_noise(noise_seed=random.randint(1, 2**64))\\n\\n    basicguider = NODE_CLASS_MAPPINGS[\\\"BasicGuider\\\"]()\\n    basicscheduler = NODE_CLASS_MAPPINGS[\\\"BasicScheduler\\\"]()\\n    samplercustomadvanced = NODE_CLASS_MAPPINGS[\\\"SamplerCustomAdvanced\\\"]()\\n    vaedecode = VAEDecode()\\n    saveimage = SaveImage()\\n\\nrouter = APIRouter(\\n    prefix=\\\"/img\\\",\\n    tags=[\\\"flux_generation\\\"],\\n    dependencies=[],\\n    responses={404: {\\\"description\\\": \\\"Not found\\\"}},\\n)\\n\\n@router.get(\\\"/flux_generation\\\")\\nasync def flux_generation(user_id:str=\\\"zxcvbnm\\\", prompt:str=\\\"a cute cat\\\", size_str: str=\\\"(1024,1024)\\\", samples: int=4, style: str=\\\"Anime Style\\\") -> ImageResponse:\\n    async def image_streamer(user_id:str, prompt:str, size_str:str, samples:int, style:str) -> AsyncGenerator[bytes, None]:\\n        # try:\\n        save_path_format = '/root/autodl-tmp/flux_data/user/{user_id}/txt2img/'\\n        save_path = save_path_format.format(user_id=user_id)\\n        os.makedirs(os.path.join(save_path), exist_ok=True)\\n        img_names = await generate_image(save_path, prompt, size_str, samples, style)\\n        images_url: List[str] = []\\n        for name in img_names:\\n            url = fileModel.get_type_file_url(user_id=user_id, fileType=\\\"txt2img\\\", filename=name)\\n            images_url.append(url)\\n        result = ImageResponse()\\n        result.data = images_url\\n        result.code = 200\\n        result.message = \\\"success\\\"\\n        return result\\n        # except:\\n        #     result = ImageResponse()\\n        #     result.data = []\\n        #     result.code = 500\\n        #     result.message = \\\"Model inference Error!\\\"\\n        #     return result\\n\\n\\n    return await image_streamer(user_id, prompt, size_str, samples, style)\\n\\nasync def generate_image(save_path:str, prompt:str, size_str:str, samples:int, style:str) -> List[str]:\\n    prompt = prompt + \\\",\\\" + image_style[style]\\n    size = ast.literal_eval(size_str)\\n    # 在这里调用你的图像生成逻辑\\n    with torch.no_grad():\\n        emptylatentimage = EmptyLatentImage()\\n        emptylatentimage_5 = emptylatentimage.generate(\\n            width=size[0], height=size[1], batch_size=samples\\n        )\\n\\n        string_literal = NODE_CLASS_MAPPINGS[\\\"String Literal\\\"]()\\n        string_literal_28 = string_literal.get_string(string=prompt)\\n\\n        cliptextencode = CLIPTextEncode()\\n        cliptextencode_6 = cliptextencode.encode(\\n            text=get_value_at_index(string_literal_28, 0),\\n            clip=get_value_at_index(dualcliploader_61, 0),\\n        )\\n\\n        basicguider_22 = basicguider.get_guider(\\n            model=get_value_at_index(unetloader_12, 0),\\n            conditioning=get_value_at_index(cliptextencode_6, 0),\\n        )\\n\\n        basicscheduler_17 = basicscheduler.get_sigmas(\\n            scheduler=\\\"simple\\\",\\n            steps=20,\\n            denoise=1,\\n            model=get_value_at_index(unetloader_12, 0),\\n        )\\n\\n        samplercustomadvanced_13 = samplercustomadvanced.sample(\\n            noise=get_value_at_index(randomnoise_25, 0),\\n            guider=get_value_at_index(basicguider_22, 0),\\n            sampler=get_value_at_index(ksamplerselect_16, 0),\\n            sigmas=get_value_at_index(basicscheduler_17, 0),\\n            latent_image=get_value_at_index(emptylatentimage_5, 0),\\n        )\\n\\n        vaedecode_8 = vaedecode.decode(\\n            samples=get_value_at_index(samplercustomadvanced_13, 0),\\n            vae=get_value_at_index(vaeloader_10, 0),\\n        )\\n        unique_id = uuid.uuid4()\\n        saveimage_9 = saveimage.save_images(\\n            filename_prefix=f\\\"{unique_id}\\\", images=get_value_at_index(vaedecode_8, 0)\\n        )\\n\\n        img_names = []\\n        for i in range(samples):\\n            img_name = saveimage_9['ui']['images'][i]['filename']\\n            origin_save_path = os.path.join(OUTPUT_PATH, img_name)\\n            shutil.move(origin_save_path, save_path)\\n            \\n            img_names.append(img_name) \\n        return img_names\\n\\n\\n\\\"\\\"\\\"\\n@router.get(\\\"/flux_generation\\\")\\nasync def flux_generation(prompt: str=\\\"a cat in the forest\\\"):\\n    with torch.no_grad():\\n        string_literal = NODE_CLASS_MAPPINGS[\\\"String Literal\\\"]()\\n        string_literal_28 = string_literal.get_string(\\n            string=prompt # \\\"a naked beauty is showerning in the bathroom\\\"\\n        )\\n        cliptextencode = CLIPTextEncode()\\n        cliptextencode_6 = cliptextencode.encode(\\n            text=get_value_at_index(string_literal_28, 0),\\n            clip=get_value_at_index(dualcliploader_61, 0),\\n        )\\n\\n        try:\\n            img_paths = []\\n            for q in range(1):\\n                basicguider_22 = basicguider.get_guider(\\n                    model=get_value_at_index(unetloader_12, 0),\\n                    conditioning=get_value_at_index(cliptextencode_6, 0),\\n                )\\n\\n                basicscheduler_17 = basicscheduler.get_sigmas(\\n                    scheduler=\\\"simple\\\",\\n                    steps=25,\\n                    denoise=1,\\n                    model=get_value_at_index(unetloader_12, 0),\\n                )\\n\\n                samplercustomadvanced_13 = samplercustomadvanced.sample(\\n                    noise=get_value_at_index(randomnoise_25, 0),\\n                    guider=get_value_at_index(basicguider_22, 0),\\n                    sampler=get_value_at_index(ksamplerselect_16, 0),\\n                    sigmas=get_value_at_index(basicscheduler_17, 0),\\n                    latent_image=get_value_at_index(emptylatentimage_5, 0),\\n                )\\n\\n                vaedecode_8 = vaedecode.decode(\\n                    samples=get_value_at_index(samplercustomadvanced_13, 0),\\n                    vae=get_value_at_index(vaeloader_10, 0),\\n                )\\n\\n\\n                saveimage_9 = saveimage.save_images(\\n                    filename_prefix=\\\"MarkuryFLUX\\\", images=get_value_at_index(vaedecode_8, 0)\\n                )\\n\\n                torch.cuda.empty_cache()\\n\\n                print(f\\\"saveimage_9: {saveimage_9}\\\")\\n                # saveimage_9: {'ui': {'images': [{'filename': 'ComfyUI_00015_.png', 'subfolder': '', 'type': 'output'}]}}\\n                img_paths.append(saveimage_9['ui']['images'][0]['filename'])\\n                # 将生成的图片路径转换为 FileResponse 列表\\n                file_responses: List[FileResponse] = []\\n\\n                for path in img_paths:\\n                    save_path = os.path.join(OUTPUT_PATH, path)\\n                    file_responses.append(FileResponse(save_path))\\n            print(img_paths)\\n            return file_responses[0]\\n            \\n        except:\\n            result = ImageResponse()\\n            result.data = []\\n            result.code = 500\\n            result.message = \\\"Model inference Error!\\\"\\n            return result\\n        \\n        \\n\\\"\\\"\\\"\\n\\\"\\\"\\\"\\nif __name__ == \\\"__main__\\\":\\n    main()\\n\\\"\\\"\\\"\\n\\nimport json\\nimport torch\\nimport random\\nimport os\\nfrom typing import List\\nfrom fastapi import APIRouter, UploadFile, File\\nfrom fastapi.responses import FileResponse\\n\\nimport os\\nfrom routers import ImageResponse, SAVE_PATH, OUTPUT_PATH\\nfrom routers import get_value_at_index\\n\\n\\nfrom nodes import (\\n    CheckpointLoaderSimple,\\n    CLIPTextEncode,\\n    NODE_CLASS_MAPPINGS,\\n    LoadImage,\\n    KSampler,\\n    SaveImage,\\n    VAEEncode,\\n    VAEDecode,\\n)\\nrouter = APIRouter(\\n    prefix=\\\"/product\\\",\\n    tags=[\\\"photography\\\"],\\n    dependencies=[],\\n    responses={404: {\\\"description\\\": \\\"Not found\\\"}},\\n)\\n\\n@router.post(\\\"/product_photography\\\")\\nasync def photography(prompt: str, myimage: UploadFile = File(...)):\\n    # 将上传的图片保存到本地\\n    if not os.path.exists(SAVE_PATH):\\n        os.makedirs(SAVE_PATH)\\n    temp_image_path = os.path.join(SAVE_PATH, myimage.filename)\\n    with open(temp_image_path, \\\"wb\\\") as buffer:\\n        buffer.write(myimage.file.read())\\n\\n    with torch.inference_mode():\\n        checkpointloadersimple = CheckpointLoaderSimple()\\n        checkpointloadersimple_4 = checkpointloadersimple.load_checkpoint(\\n            ckpt_name=\\\"epicrealism_naturalSinRC1VAE.safetensors\\\"\\n        )\\n\\n        cliptextencode = CLIPTextEncode()\\n        cliptextencode_6 = cliptextencode.encode(\\n            text= prompt, # \\\"in a stone on the beach\\\",\\n            clip=get_value_at_index(checkpointloadersimple_4, 1),\\n        )\\n\\n        cliptextencode_7 = cliptextencode.encode(\\n            text=\\\"text, watermark\\\", clip=get_value_at_index(checkpointloadersimple_4, 1)\\n        )\\n\\n        loadimage = LoadImage()\\n        loadimage_15 = loadimage.load_image(image=myimage.filename)\\n\\n        imageresize = NODE_CLASS_MAPPINGS[\\\"ImageResize+\\\"]()\\n        imageresize_16 = imageresize.execute(\\n            width=1000,\\n            height=600,\\n            interpolation=\\\"nearest\\\",\\n            method=\\\"stretch\\\",\\n            condition=\\\"always\\\",\\n            multiple_of=0,\\n            image=get_value_at_index(loadimage_15, 0),\\n        )\\n\\n        vaeencode = VAEEncode()\\n        vaeencode_14 = vaeencode.encode(\\n            pixels=get_value_at_index(imageresize_16, 0),\\n            vae=get_value_at_index(checkpointloadersimple_4, 2),\\n        )\\n\\n        iclightconditioning = NODE_CLASS_MAPPINGS[\\\"ICLightConditioning\\\"]()\\n        iclightconditioning_12 = iclightconditioning.encode(\\n            multiplier=0.18215,\\n            positive=get_value_at_index(cliptextencode_6, 0),\\n            negative=get_value_at_index(cliptextencode_7, 0),\\n            vae=get_value_at_index(checkpointloadersimple_4, 2),\\n            foreground=get_value_at_index(vaeencode_14, 0),\\n        )\\n\\n        createshapemask = NODE_CLASS_MAPPINGS[\\\"CreateShapeMask\\\"]()\\n        createshapemask_17 = createshapemask.createshapemask(\\n            shape=\\\"circle\\\",\\n            frames=1,\\n            location_x=850,\\n            location_y=125,\\n            grow=0,\\n            frame_width=get_value_at_index(imageresize_16, 1),\\n            frame_height=get_value_at_index(imageresize_16, 2),\\n            shape_width=256,\\n            shape_height=512,\\n        )\\n\\n        growmaskwithblur = NODE_CLASS_MAPPINGS[\\\"GrowMaskWithBlur\\\"]()\\n        growmaskwithblur_18 = growmaskwithblur.expand_mask(\\n            expand=0,\\n            incremental_expandrate=0,\\n            tapered_corners=True,\\n            flip_input=False,\\n            blur_radius=12,\\n            lerp_alpha=1,\\n            decay_factor=1,\\n            fill_holes=False,\\n            mask=get_value_at_index(createshapemask_17, 0),\\n        )\\n\\n        remapmaskrange = NODE_CLASS_MAPPINGS[\\\"RemapMaskRange\\\"]()\\n        remapmaskrange_19 = remapmaskrange.remap(\\n            min=0,\\n            max=0.7000000000000001,\\n            mask=get_value_at_index(growmaskwithblur_18, 0),\\n        )\\n\\n        masktoimage = NODE_CLASS_MAPPINGS[\\\"MaskToImage\\\"]()\\n        masktoimage_20 = masktoimage.mask_to_image(\\n            mask=get_value_at_index(remapmaskrange_19, 0)\\n        )\\n\\n        vaeencode_21 = vaeencode.encode(\\n            pixels=get_value_at_index(masktoimage_20, 0),\\n            vae=get_value_at_index(checkpointloadersimple_4, 2),\\n        )\\n\\n        loadandapplyiclightunet = NODE_CLASS_MAPPINGS[\\\"LoadAndApplyICLightUnet\\\"]()\\n        ksampler = KSampler()\\n        vaedecode = VAEDecode()\\n        saveimage = SaveImage()\\n\\n        try:\\n            img_paths = []\\n            for q in range(1):\\n                loadandapplyiclightunet_13 = loadandapplyiclightunet.load(\\n                    model_path=\\\"iclight_sd15_fc_unet_ldm.safetensors\\\",\\n                    model=get_value_at_index(checkpointloadersimple_4, 0),\\n                )\\n\\n                ksampler_3 = ksampler.sample(\\n                    seed=random.randint(1, 2**64),\\n                    steps=30,\\n                    cfg=3,\\n                    sampler_name=\\\"dpmpp_2m_sde\\\",\\n                    scheduler=\\\"karras\\\",\\n                    denoise=1,\\n                    model=get_value_at_index(loadandapplyiclightunet_13, 0),\\n                    positive=get_value_at_index(iclightconditioning_12, 0),\\n                    negative=get_value_at_index(iclightconditioning_12, 1),\\n                    latent_image=get_value_at_index(vaeencode_21, 0),\\n                )\\n\\n                vaedecode_8 = vaedecode.decode(\\n                    samples=get_value_at_index(ksampler_3, 0),\\n                    vae=get_value_at_index(checkpointloadersimple_4, 2),\\n                )\\n\\n                saveimage_9 = saveimage.save_images(\\n                    filename_prefix=\\\"ComfyUI\\\", images=get_value_at_index(vaedecode_8, 0)\\n                )\\n                # print(f\\\"saveimage_9: {saveimage_9}\\\")\\n                # saveimage_9: {'ui': {'images': [{'filename': 'ComfyUI_00015_.png', 'subfolder': '', 'type': 'output'}]}}\\n                img_paths.append(saveimage_9['ui']['images'][0]['filename'])\\n            \\n            # 将生成的图片路径转换为FileResponse列表\\n            images_url: List[str] = []\\n            # 将生成的图片路径转换为 FileResponse 列表\\n            file_responses: List[FileResponse] = []\\n\\n            for path in img_paths:\\n                save_path = os.path.join(OUTPUT_PATH, path)\\n                file_responses.append(FileResponse(save_path))\\n            #     url = f'https://u447318-95cc-7e79cc31.bjc1.seetacloud.com:8443/docs#/photography/photography_photography_product_photography_post?filename={save_path}'\\n            #     images_url.append(url)\\n            # result = ImageResponse()\\n            # result.data = images_url\\n            # result.code = 200\\n            # result.message = \\\"success\\\"\\n            # return result\\n            return file_responses[0]\\n            \\n        except:\\n            result = ImageResponse()\\n            result.data = []\\n            result.code = 500\\n            result.message = \\\"Model inference Error!\\\"\\n            return result\\n\\n\\n\\nimport os\\nimport random\\nimport sys\\nfrom typing import Sequence, Mapping, Any, Union, List\\nimport torch\\nfrom fastapi import APIRouter, UploadFile, File\\nfrom fastapi.responses import FileResponse\\n\\nimport os\\nfrom routers import ImageResponse, SAVE_PATH, OUTPUT_PATH\\nfrom routers import  get_value_at_index\\n\\n\\nfrom nodes import (\\n    KSampler,\\n    SaveImage,\\n    NODE_CLASS_MAPPINGS,\\n    VAEDecode,\\n    CheckpointLoaderSimple,\\n    EmptyLatentImage,\\n    CLIPTextEncode,\\n    LoadImage,\\n)\\n\\nrouter = APIRouter(\\n    prefix=\\\"/product\\\",\\n    tags=[\\\"enhancement\\\"],\\n    dependencies=[],\\n    responses={404: {\\\"description\\\": \\\"Not found\\\"}},\\n)\\n\\n@router.post(\\\"/product_enhancement\\\")\\nasync def product_enhancement(prompt: str, myimage: UploadFile = File(...)):\\n    # 将上传的图片保存到本地\\n    if not os.path.exists(SAVE_PATH):\\n        os.makedirs(SAVE_PATH)\\n    temp_image_path = os.path.join(SAVE_PATH, myimage.filename)\\n    with open(temp_image_path, \\\"wb\\\") as buffer:\\n        buffer.write(myimage.file.read())\\n       \\n    with torch.inference_mode():\\n        checkpointloadersimple = CheckpointLoaderSimple()\\n        checkpointloadersimple_4 = checkpointloadersimple.load_checkpoint(\\n            ckpt_name=\\\"epicrealism_naturalSinRC1VAE.safetensors\\\"\\n        )\\n\\n        cliptextencode = CLIPTextEncode()\\n        cliptextencode_6 = cliptextencode.encode(\\n            text= prompt, # \\\"silk sheets, wooden, flowers, \\\",\\n            clip=get_value_at_index(checkpointloadersimple_4, 1),\\n        )\\n\\n        cliptextencode_7 = cliptextencode.encode(\\n            text=\\\"lowres, bad anatomy, bad hands, cropped, worst quality\\\",\\n            clip=get_value_at_index(checkpointloadersimple_4, 1),\\n        )\\n\\n        loadimage = LoadImage()\\n        loadimage_11 = loadimage.load_image(image=myimage.filename)\\n\\n        imageresize = NODE_CLASS_MAPPINGS[\\\"ImageResize+\\\"]()\\n        imageresize_14 = imageresize.execute(\\n            width=1024,\\n            height=1024,\\n            interpolation=\\\"nearest\\\",\\n            method=\\\"keep proportion\\\",\\n            condition=\\\"always\\\",\\n            multiple_of=0,\\n            image=get_value_at_index(loadimage_11, 0),\\n        )\\n\\n        easy_imagerembg = NODE_CLASS_MAPPINGS[\\\"easy imageRemBg\\\"]()\\n        easy_imagerembg_12 = easy_imagerembg.remove(\\n            rem_mode=\\\"RMBG-1.4\\\",\\n            image_output=\\\"Preview\\\",\\n            save_prefix=\\\"ComfyUI\\\",\\n            torchscript_jit=False,\\n            images=get_value_at_index(imageresize_14, 0),\\n        )\\n\\n        iclightapplymaskgrey = NODE_CLASS_MAPPINGS[\\\"ICLightApplyMaskGrey\\\"]()\\n        iclightapplymaskgrey_41 = iclightapplymaskgrey.apply_mask(\\n            image=get_value_at_index(easy_imagerembg_12, 0),\\n            alpha=get_value_at_index(easy_imagerembg_12, 1),\\n        )\\n\\n        vaeencodeargmax = NODE_CLASS_MAPPINGS[\\\"VAEEncodeArgMax\\\"]()\\n        vaeencodeargmax_24 = vaeencodeargmax.encode(\\n            pixels=get_value_at_index(iclightapplymaskgrey_41, 0),\\n            vae=get_value_at_index(checkpointloadersimple_4, 2),\\n        )\\n\\n        emptylatentimage = EmptyLatentImage()\\n        emptylatentimage_42 = emptylatentimage.generate(\\n            width=get_value_at_index(imageresize_14, 1),\\n            height=get_value_at_index(imageresize_14, 2),\\n            batch_size=1,\\n        )\\n\\n        vaedecode = VAEDecode()\\n        vaedecode_43 = vaedecode.decode(\\n            samples=get_value_at_index(emptylatentimage_42, 0),\\n            vae=get_value_at_index(checkpointloadersimple_4, 2),\\n        )\\n\\n        splitimagewithalpha = NODE_CLASS_MAPPINGS[\\\"SplitImageWithAlpha\\\"]()\\n        splitimagewithalpha_47 = splitimagewithalpha.split_image_with_alpha(\\n            image=get_value_at_index(easy_imagerembg_12, 0)\\n        )\\n\\n        imagecompositemasked = NODE_CLASS_MAPPINGS[\\\"ImageCompositeMasked\\\"]()\\n        imagecompositemasked_46 = imagecompositemasked.composite(\\n            x=0,\\n            y=0,\\n            resize_source=False,\\n            destination=get_value_at_index(vaedecode_43, 0),\\n            source=get_value_at_index(splitimagewithalpha_47, 0),\\n            mask=get_value_at_index(easy_imagerembg_12, 1),\\n        )\\n\\n        vaeencodeargmax_37 = vaeencodeargmax.encode(\\n            pixels=get_value_at_index(imagecompositemasked_46, 0),\\n            vae=get_value_at_index(checkpointloadersimple_4, 2),\\n        )\\n\\n        iclightconditioning = NODE_CLASS_MAPPINGS[\\\"ICLightConditioning\\\"]()\\n        iclightconditioning_62 = iclightconditioning.encode(\\n            multiplier=0.18215,\\n            positive=get_value_at_index(cliptextencode_6, 0),\\n            negative=get_value_at_index(cliptextencode_7, 0),\\n            vae=get_value_at_index(checkpointloadersimple_4, 2),\\n            foreground=get_value_at_index(vaeencodeargmax_24, 0),\\n        )\\n\\n        loadandapplyiclightunet = NODE_CLASS_MAPPINGS[\\\"LoadAndApplyICLightUnet\\\"]()\\n        easy_ipadapterapply = NODE_CLASS_MAPPINGS[\\\"easy ipadapterApply\\\"]()\\n        ksampler = KSampler()\\n        saveimage = SaveImage()\\n        detailtransfer = NODE_CLASS_MAPPINGS[\\\"DetailTransfer\\\"]()\\n\\n        try:\\n            img_paths = []\\n            for q in range(1):\\n                loadandapplyiclightunet_61 = loadandapplyiclightunet.load(\\n                    model_path=\\\"iclight_sd15_fc_unet_ldm.safetensors\\\",\\n                    model=get_value_at_index(checkpointloadersimple_4, 0),\\n                )\\n\\n                easy_ipadapterapply_58 = easy_ipadapterapply.apply(\\n                    preset=\\\"PLUS (high strength)\\\",\\n                    lora_strength=0.6,\\n                    provider=\\\"CPU\\\",\\n                    weight=1,\\n                    weight_faceidv2=1,\\n                    start_at=0,\\n                    end_at=1,\\n                    cache_mode=\\\"all\\\",\\n                    use_tiled=False,\\n                    model=get_value_at_index(loadandapplyiclightunet_61, 0),\\n                    image=get_value_at_index(imagecompositemasked_46, 0),\\n                    attn_mask=get_value_at_index(easy_imagerembg_12, 1),\\n                )\\n\\n                ksampler_16 = ksampler.sample(\\n                    seed=random.randint(1, 2**64),\\n                    steps=25,\\n                    cfg=2,\\n                    sampler_name=\\\"dpmpp_2m_sde\\\",\\n                    scheduler=\\\"karras\\\",\\n                    denoise=0.9,\\n                    model=get_value_at_index(easy_ipadapterapply_58, 0),\\n                    positive=get_value_at_index(iclightconditioning_62, 0),\\n                    negative=get_value_at_index(iclightconditioning_62, 1),\\n                    latent_image=get_value_at_index(vaeencodeargmax_37, 0),\\n                )\\n\\n                vaedecode_17 = vaedecode.decode(\\n                    samples=get_value_at_index(ksampler_16, 0),\\n                    vae=get_value_at_index(checkpointloadersimple_4, 2),\\n                )\\n\\n                saveimage_18 = saveimage.save_images(\\n                    filename_prefix=\\\"ComfyUI\\\", images=get_value_at_index(vaedecode_17, 0)\\n                )\\n\\n                detailtransfer_51 = detailtransfer.process(\\n                    mode=\\\"add\\\",\\n                    blur_sigma=1,\\n                    blend_factor=1,\\n                    target=get_value_at_index(vaedecode_17, 0),\\n                    source=get_value_at_index(splitimagewithalpha_47, 0),\\n                )\\n\\n                saveimage_52 = saveimage.save_images(\\n                    filename_prefix=\\\"ComfyUI\\\",\\n                    images=get_value_at_index(detailtransfer_51, 0),\\n                )\\n                # print(f\\\"saveimage_52: {saveimage_52}\\\")\\n                # saveimage_9: {'ui': {'images': [{'filename': 'ComfyUI_00015_.png', 'subfolder': '', 'type': 'output'}]}}\\n                img_paths.append(saveimage_52['ui']['images'][0]['filename'])\\n                \\n                # 将生成的图片路径转换为 FileResponse 列表\\n                file_responses: List[FileResponse] = []\\n\\n                for path in img_paths:\\n                    save_path = os.path.join(OUTPUT_PATH, path)\\n                    file_responses.append(FileResponse(save_path))\\n                return file_responses[0]\\n            \\n        except:\\n            result = ImageResponse()\\n            result.data = []\\n            result.code = 500\\n            result.message = \\\"Model inference Error!\\\"\\n            return result\",\"difficulty\":\"easy\",\"domain\":\"Code Repository Understanding\",\"length\":\"short\",\"question\":\"This codebase implements a method for generating stylized narrative images from character portraits. What is the overall process of this method?\",\"sub_domain\":\"Code repo QA\"}","display_format":"text","language":"","answer_status":"published","assets":[],"source_url":"https://huggingface.co/datasets/zai-org/LongBench-v2","history":"initial import","indexing_mode":"noindex","subproblems":[],"grids":[]}