"Actual" upscaling support in the resize filter

[Stoppedpuma]

This may be out of scope / unwanted so I'm pinging you @woelper directly to think about this. I personally think this is a huge plus to have but you may think otherwise.

In this case, "actual" means with an actual algorithm for this. I've personally used a tool on flathub called Upscaler and I've had some absolutely amazing results with it over the past year or so that I've used this program. Under the hood, it uses an algorithm called Real-ESRGAN ncnn Vulkan which happens to have multiple models built in.

Here's a close up side by side of an image upscaled from 1024x683 to 4096x2732 with Upscaler and the same image scaled up with the resize filter:

There's some more examples on the Real-ESRGAN ncnn Vulkan repo.

A thing which could be nice here is allowing the user to select their own models similar to how LUTs currently work. Some other similar tools like upscayl have their own custom models which would be cool to allow users to import their own, this same tool even has instructions on model conversion to create your own!

[woelper]

Interesting idea!

Let me know if you discover any rust libraries capable of that. I have found this: https://github.com/millardjn/rusty_sr

I am not sure about the required models. They may be large and have incompatible licenses.

[B-LechCode]

What about using: https://docs.rs/onnxruntime/latest/onnxruntime/ and add the according neural network to apply as .onnx? The base repo, as far I could tell, uses PyTorch, which we could use to export to onnx - or maybe there's already an onnx file for this net.
This would enable using other networks with little effort, too.

Using neural networks for upscaling would imply some research on usable resoultions and other solutions available :)

[Stoppedpuma]

What about using: https://docs.rs/onnxruntime/latest/onnxruntime/ and add the according neural network to apply as .onnx? The base repo, as far I could tell, uses PyTorch, which we could use to export to onnx - or maybe there's already an onnx file for this net. This would enable using other networks with little effort, too.

Using neural networks for upscaling would imply some research on usable resoultions and other solutions available :)

I'll look into this approach soon! This would actually work out very well as PyTorch is very accessible and I'm pretty sure the dominant machine learning library IIRC.