Training perceiver for image/ video features

[sauradip]

Can I train this perceiver with deeper features of an image from a pretrained CNN and use it as input for classification ? Has anyone tried it ? I don't seem to get good result during inference ! Can anyone help me out ?

[pcicales]

@sauradip Yes, in fact the perceiver is really better suited to deal with input feature embeddings, not raw input signals. I would play with your optimizer and learning rate, these types of architectures are sometimes sensitive to hyperparameters.