Can we use LoftQ to optimize vision foundation models like OWL-ViT v2 and Grounding Dino?

[solomonmanuelraj]

Hi Team,

I need your help to optimize VFMs like OWL-ViT v2 and Grounding Dino to reduce the memory size and deploy in the edge device.

Your feedback and reference code will be helpful.

with thanks

[tourzhao]

You can definitely apply LoftQ. to any other transformer-based foundation models. The code should be straightforward to be adapted from our provided code for LLM.