Does spspmm operation support autograd?

[changym3]

Hi, you say autograd is supported for values tensors, but it seems it doesn't work in spspmm.

Like this:

indexA = torch.tensor([[0, 0, 1, 2, 2], [1, 2, 0, 0, 1]])
valueA = torch.tensor([1, 2.0, 3, 4, 5], requires_grad=True)
indexB = torch.tensor([[0, 2], [1, 0]])
valueB = torch.tensor([2, 4.0], requires_grad=True)
indexC, valueC = torch_sparse.spspmm(indexA, valueA, indexB, valueB, 3, 3, 2)

print(valueC.requires_grad)
print(valueC.grad_fn)

And the answer is:

False
None

In my case, I want to parameterize the sparse adjacent matrix and feature matrix in GCN, so the inputs need to be both differentiable. I wonder if there're some bugs or just the way it is.

Regards.

[rusty1s]

That's the only function that does not have proper autograd support. Gradients for sparse-sparse matrix multiplication are quite difficult to obtain (since they are usually dense). I had a working, but slow implementation up to 0.4.4 release, but removed it since it wasn't a really good implementation. If you desperately need it, feel free to try it out.

[changym3]

That's the only function that does not have proper autograd support. Gradients for sparse-sparse matrix multiplication are quite difficult to obtain (since they are usually dense). I had a working, but slow implementation up to 0.4.4 release, but removed it since it wasn't a really good implementation. If you desperately need it, feel free to try it out.

Hey! Thanks for your great work! I have installed the 0.4.4 release of torch_sparse and it totally works out in my experiments! Maybe you could add this information to the documentation. It takes me so long to figure out this no-autograd problem.

Thanks a lot again!

[LuciusMos]

Hey! Thanks for your great work! I have installed the 0.4.4 release of torch_sparse and it totally works out in my experiments! Maybe you could add this information to the documentation. It takes me so long to figure out this no-autograd problem.

Thanks a lot again!

Thank you so much for your question raising! It really troubles me for almost a week!

[rusty1s]

Sorry for the inconveniences. I have plans to add backward support for spspmm back in ASAP, see pyg-team/pytorch_geometric#1465.

[jlevy44]

Do you have any updates on autograd support?

[jlevy44]

I'm parameterizing the weights of a sparse matrix to treat it as a locally connected network for a sparsely connected MLP implementation. Could I still run a backward pass to update these weights after calling matmul between this sparse matrix and a dense input?

[jlevy44]

Nevermind, already seeing some nice implementations out there!
https://pypi.org/project/sparselinear/
https://stackoverflow.com/questions/63893602/neural-network-layer-without-all-connections

[JRD971000]

Does spspmm still lack autograd support?

[github-actions]

This issue had no activity for 6 months. It will be closed in 2 weeks unless there is some new activity. Is this issue already resolved?

[jaynanavati-az]

Does spspmm still lack autograd support? @rusty1s .. it seems to use SparseTensor, which is supposed to be fully supported by autograd?