How to override ApproximateGP forward method ?

[tardybe]

Hello,

It seems that the signature for the forward method for Approximate GP cannot be overridden.

Indeed, the Approximate GP class overrides the __call__ method from module. This override requires the forward method to have a parameter named inputs as single parameters.

Then it seems not possible to create a custom class where several inputs are given to the forward method.

I'm not sure I understand why it is imposed to have a single tensor as input. Because with most pytorch classes there is a lot of freedom to easily adapt existing classes to different projects.

Is it a constraint that the class only uses variational_distribution based on covariance and mean values.

If we want to overload the signature of the forward method, it seems that we should also overload the functions of the variational strategies?

I may have missed some information.

[wjmaddox]

In general, yes, GPyTorch typically assumes that the inputs are a single tensor (usually of shape n x ....) rather than a more general object. This is because most of the commonly used kernels are really only well defined for inputs that are real valued.

It should be possible to get a version working with another data type (a list or something else like that) by overwriting the forwards method of (exact or approximate) gp and writing specialized means and kernels.

Is there a specific data type that you're trying to use here?

[tardybe]

Thank you for your answer.

We have a processing chain for land cover mapping which can use torch models. The chain allow to use several remote sensing images from different sensors (Sentinel2, Sentinel1, Landsat...). For all models pytorch we have the same signature for the forward method:

def forward(s2, s2masks, s1, ...):
      pass

Then according to the sensor provided (the others are set to None by default) the processing in forward can differ.

Ideally we want to do the same thing for gpytorch models, but as far I understand, the forward method should concatenate all inputs as an unique tensor for the use of existing kernels.

[wjmaddox]

So are these inputs all combined together in the kernel in some manner or is the model additive across these types of inputs?

[tardybe]

Hello, sorry for the response delay.

For our use case, the signature of the function would not necessarily force the number of kernels. We would like a function with more freedom.

I will try to code a class equivalent to ApproximateGP but without this constraint on the forward signature to fit in a more complex workflow.

[gpleiss]

We would like a function with more freedom.

Can you describe exactly what you mean by "more freedom"? Do you want a variable number of inputs?

What I would maybe suggest is that you concatenate s2, s2masks, s1, ... along the -1 dimension. In forward, you can then "unstack" this input, and then pass each item into its separate kernel function.

def forward(self, inputs):
s2, s2masks, s1, *others = inputs.split([1, 1, 1, -1], dim=-1)
# ...