Little difference between speed for legacy vs tensor based TSDF integration

[salmanmaq]

Hi. I tried out both the legacy and tensor based TSDF integration pipelines. My workflow is something like I shared here: #4655

I am a using Open3D 0.14.1 on WSL2 and I have an intel 7th gen i7 with GTX 1070. For a scene with 828 frames, the average time I get for a scene reconstruction with the tensor/VoxelBlockGrid based integration is ~42 seconds. The time for the legacy system is around 45-48 seconds. The other overlapping variables are kept the same. I was expecting a larger time difference between the two. Is there something I am missing to understand?

Thanks

[salmanmaq]

I did some more tests and here are the numbers:

Comparison of legacy and tensor pipeline wrt speed and memory usage (variables: number of frames, voxel size):
Using a video with 8976 frames with a voxel size of 1.5 cm, the legacy pipeline took 436 seconds, the tensor pipeline took 404 seconds (~5-6 GB GPU memory usage).
Using a video with 828 frames with a voxel size of 1.5 cm, the legacy pipeline took 42 seconds (peak 2 GB RAM usage), the tensor pipeline took 37 seconds (~5 GB GPU memory usage).
Using a video with 8976 frames with a voxel size of 0.586 cm (3/512 m), the legacy pipeline crashed as it ran out of memory (16+ GB RAM usage), the tensor pipeline crashed due to CUDA out of memory error (8+ GB GPU memory usage)
Using a video with 828 frames with a voxel size of 0.586 cm (3/512 m), the legacy pipeline took 178 seconds (peak 11.1 GB RAM usage), the tensor pipeline took 40 seconds (~6-7 GB GPU memory usage)

Even if the voxel size increases, the tensor pipeline runs in more or less the same time which is not the case for the legacy pipeline.

[theNded]

Your workflow has an overhead for input data conversion. How much does that cost?
Also, depth_max=5.0 is unreasonably large for the typical usage. depth_max=3.0 is more preferred, if you frequently run into memory issues.

[salmanmaq]

Thank you for the reply. I will try to get some numbers over the weekend.
For depth_max, we are trying to set it as the range for the sensor (which now that you mentioned it also seems like a bad idea). I will try with values of 3.0.

By the way, are the depth_max and depth_trunc`sdf_trunc` parameters in tensor and legacy pipelines respectively functionally equivalent i.e. they cap the maximum depth?

[theNded]

depth_max caps the maximum depth; the other two control the surface thickness during integration, which is recommended to be 5x voxel size in integration.

[salmanmaq]

I tried to profile my code and it looks like the data loading + pre-processing overhead is a taking a lot of time! I have so far only been able to test the legacy pipeline but for a scene with 828 frames, the total reconstruction time is around 45 seconds. The RGB image resizing was taking around 30 seconds of this and there's a room for lot of optimizations there. I am hoping to bring the time for the whole thing down to 15 seconds with some relatively easy modifications. And the gains would be much more when there are larger scenes with ~9000 frames! Thanks a lot for pointing me in this direction. I will try to post updated numbers soon.

[salmanmaq]

Optimizing parts of data loading and pre-processing, I was able to get much better numbers:

1. 4570 frames processed in 92 seconds (vs 225 seconds before)
2. 9877 frames processed in 196 seconds (vs 436 seconds before)
3. 829 frames processed in 23 seconds (vs 42 seconds before)