Leverage cuda acceleration in the sensing perception pipeline

[knzo25]

Checklist

• I've read the contribution guidelines.
• I've searched other issues and no duplicate issues were found.
• I've agreed with the maintainers that I can plan this task.

Description

Background:
We have long struggled with the latency in our sensing/perception pipeline. Even though we use very rudimentary algorithms, the latency is considerable enough that we can not opt to use more complex ones due to fear of increasing reaction times.
In the same vein, autoware has proven to be too heavy for new sensors with higher data rates and we have had to resort to down sample approaches to bound the execution times, even though using the new sensors fully holds the potential to increase the overall performance of autoware.

Previous conversations:

• In the sensing / perception WG, the idea of acceleration and some PoCs were discussed. There was no opposition other than compatibility and granularity when possible.
• A similar feature request was mentioned in https://github.com/orgs/autowarefoundation/discussions/5396

Purpose

Accelerate the sensing/perception pipeline via cuda-based acceleration

Possible approaches

• Accelerate the pointcloud preprocessing pipeline
• Accelerate nodes that use the pointcloud in the perception pipeline
• Implement a transport layer for CUDA so that we can avoid host->device->host copies whenever possible

Definition of done

PRs that implement

• acceleration of the sensing perception pipeline
• type adaptation and negotiation
  are merged to universe and/or autoware

[knzo25]

More background that is suited better in a post rather than the issue's description:
Apologies for skipping the discussion step of the contributing guidelines, but this idea was floated around our SP WG for a while.
While we did not move much in the meetings, I ended up making an implementation because I was worried that one of our projects would struggle without it.

The following PRs are part of this issue/task:

• feat(autoware_cuda_pointcloud_preprocessor): a cuda-accelerated pointcloud preprocessor #9454 (pointcloud preprocessing)
• feat(autoware_pointcloud_preprocessor): cuda-accelerated pointcloud concatenation #9455 (concatenation)
• feat(autoware_lidar_centerpoint): added the cuda_blackboard to centerpoint #9453 (centerpoint)
• transfusion (TODO)
• feat(autoware_probabilistic_occupancy_grid_map): cuda accelerated implementation #9542 (OGM - the first implementation will be independent of the blackboard to ease the transition)
• feat: acceleration and transport layer tier4/aip_launcher#348 (aip_launcher)
• feat: acceleration and transport layer sample_sensor_kit_launch#111 (sample_sensor_kit_launch)
• feat(autoware.repos): added the cuda_blackboard and negotiated to the repos file autoware#5710 (autoware)

Pending tasks:

• Adapt Transfusion to use the cuda blackboard
• Accelerate the ground filtering ( @badai-nguyen mentioned he was doing something along these lines)
• Accelerate subsample filters (@manato seemed interested)

[mitsudome-r]

@knzo25
Are you available sometime this week? I would like to have a short call about how we can transfer cuda_blackboard repo to AWF.

[knzo25]

@mitsudome-r
Thanks !
I am available at any time today and thursday.
Friday any time that my google calendar is open

[knzo25]

Update:

• The cuda_blackboard was added to autoware
• The cuda accelerated pointcloud preprocessor was merged (thanks to @amadeuszsz, @mojomex, and @manato )
• The concat node PR was split into two:
  • feat(autoware_pointcloud_preprocessor): templated version of the pointcloud concatenation #10298 (concatenation - template refactor)
  • feat(autoware_cuda_pointcloud_preprocessor): pointcloud concatenation #10300 (concatenation - cuda version)
• The test branch (with all the PRs) was changed to: https://github.com/knzo25/autoware.universe/tree/feat/cuda_acceleration_and_transport_layer2

I will make a re-check of the concat's refactor just in case, which should not affect the review process, address comments in centerpoint and make any final adjustments, since it can be merged independently thanks to the blackboard being added, and address the launcher related comments, although that PR can not be merged until the very end.