New release with IPAdapters support

[pietrobolcato]

Hey there, first of all thank you so much for the amazing work!

We are using SDXL with IPAdapters, and in the current released version (0.1.2) is not working, but we noticed that the latest commits in main add support for it.

Would it be possible to create a new release with the latest version of the code? Would very much appreciate, and make our life simpler 🙏

Thank you so much in advance! Keep up the great work!

[algoDd]

We are facing the following error with Version 0.1.2

Traceback (most recent call last):
images = hgf_pipe.tgate(
File "/workspaces/py/dist/export/python/virtualenvs/python_default/3.10.14/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
File "/workspaces/dist/export/python/virtualenvs/python_default/3.10.14/lib/python3.10/site-packages/tgate/SDXL.py", line 361, in tgate
image_embeds = self.prepare_ip_adapter_image_embeds(

TypeError: StableDiffusionXLPipeline.prepare_ip_adapter_image_embeds() missing 1 required positional argument: 'do_classifier_free_guidance'

Which seems to be solved in your main branch. It would be great if you can help us with a latest release or help us to find some other way to resolve this issue.

Thank you

[WentianZhang-ML]

Hi, thanks for your reminder!

We have released the latest version of TGATE(v1.0.0) on PyPi. Please upgrade it and try again!

[algoDd]

@WentianZhang-ML Thank you so much for a speedy release 🚀. It works perfectly for generating images but with IpAdapters it has another issue.

hidden_states, cache = tgate_processor(
File "/workspaces/dist/export/python/virtualenvs/python_default/3.10.14/lib/python3.10/site-packages/tgate/tgate_utils.py", line 189, in tgate_processor
hidden_states.shape if encoder_hidden_states is None else encoder_hidden_states.shape
AttributeError: 'tuple' object has no attribute 'shape'

Seems like you are expecting only hidden states where are UNET diffusers returns a Tuple when there are Ipadapter images the tuple basically has hidden states and image embeddings which we need for applying IpAdapterImage style.

Can you please help us in this matter?

Thank you 🙌

[WentianZhang-ML]

@WentianZhang-ML Thank you so much for a speedy release 🚀. It works perfectly for generating images but with IpAdapters it has another issue.

hidden_states, cache = tgate_processor(
File "/workspaces/dist/export/python/virtualenvs/python_default/3.10.14/lib/python3.10/site-packages/tgate/tgate_utils.py", line 189, in tgate_processor
hidden_states.shape if encoder_hidden_states is None else encoder_hidden_states.shape
AttributeError: 'tuple' object has no attribute 'shape'

Seems like you are expecting only hidden states where are UNET diffusers returns a Tuple when there are Ipadapter images the tuple basically has hidden states and image embeddings which we need for applying IpAdapterImage style.

Can you please help us in this matter?

Thank you 🙌

In this project, we only rewrote the attention process AttnProcessor2_0 in tgate_utils.py. If you want to use IPAdapters, you can try to modify the tgate_processor according to IPAdapterAttnProcessor or IPAdapterAttnProcessor2_0.

[algoDd]

Cool I adopted it and fixed it on my end let me know If it makes sense to add another pipeline in your code for IPAdapters I can open a PR 🙌

But thanks a lot for amazing work 🚀

[WentianZhang-ML]

Cool I adopted it and fixed it on my end let me know If it makes sense to add another pipeline in your code for IPAdapters I can open a PR 🙌

But thanks a lot for amazing work 🚀

Sure, please feel free to open a PR! We are happy to collaborate with all contributors!