refactor: add autoware_cuda_dependency_meta

[esteve]

Description

Added meta package that packages that depend on CUDA must depend on.

Related links

Parent Issue:

• Create a CUDA dependency meta-package #10069

How was this PR tested?

Notes for reviewers

None.

Interface changes

None.

Effects on system behavior

None.

[github-actions]

Thank you for contributing to the Autoware project!

🚧 If your pull request is in progress, switch it to draft mode.

Please ensure:

• You've checked our contribution guidelines.
• Your PR follows our pull request guidelines.
• All required CI checks pass before marking the PR ready for review.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 26.39%. Comparing base (b3caa5c) to head (77ed72d).
Report is 2 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #10073      +/-   ##
==========================================
+ Coverage   26.31%   26.39%   +0.07%     
==========================================
  Files        1384     1389       +5     
  Lines      107560   107785     +225     
  Branches    41421    41515      +94     
==========================================
+ Hits        28306    28445     +139     
- Misses      76433    76503      +70     
- Partials     2821     2837      +16     

Flag	Coverage Δ		*Carryforward flag
differential	4.55% <ø> (?)
differential-cuda	4.35% <ø> (?)
total	26.39% <ø> (+0.07%)	⬆️	Carriedforward from fbc7a9d

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[xmfcx]

Hi @esteve thanks for working on this task, is this ready for review? Are there any blockers?

[esteve]

@xmfcx I'm testing this in our buildfarm to make sure packages that depend on autoware_cuda_dependency_meta can be filtered out easily.

[esteve]

This PR is now ready for review, the buildfarm builds fine when excluding the autoware_cuda_dependency_meta autowarefoundation/autoware-deb-packages#127

[esteve]

@xmfcx this PR requires approvals from 15 people, would it be possible to merge it with only one or two? I worry it might take too long if we have to wait for 15. Thanks.

[amadeuszsz]

LGTM!
I'm not only sure if for new packages we should align package version in package.xml with latest tag. Nevertheless, new release will overwrite it.

[knzo25]

@esteve
Can you add this package as well?
https://github.com/autowarefoundation/autoware.universe/blob/main/perception/autoware_probabilistic_occupancy_grid_map/CMakeLists.txt

Or rather, a question:
The previous package contains, say, 2 different nodes that share some logic.
One needs cuda and will not be built if there is no nvcc. The other one does not need cuda so will be built no matter what.
Should we use the meta dependency in this case?

[esteve]

@knzo25 if the package depends on CUDA in one way or another, yes, it must depend on autoware_cuda_dependency_meta, even if some of its components can be built without CUDA. Eventually we want to add REQUIRED to find_package(CUDA) for those packages that depend on CUDA to make the dependency clear and so that builds fail if CUDA is not available.

[esteve]

@knzo25 I've added autoware_cuda_dependency_meta to autoware_probabilistic_occupancy_grid_map in 88ef5c0

[xmfcx]

$ colcon list --base-paths src/universe/autoware.universe --packages-above autoware_tensorrt_common autoware_cuda_utils --names-only
autoware_bytetrack
autoware_cuda_utils
autoware_detected_object_validation
autoware_detection_by_tracker
autoware_image_projection_based_fusion
autoware_lidar_apollo_instance_segmentation
autoware_lidar_centerpoint
autoware_lidar_transfusion
autoware_probabilistic_occupancy_grid_map
autoware_raindrop_cluster_filter
autoware_shape_estimation
autoware_tensorrt_classifier
autoware_tensorrt_common
autoware_tensorrt_yolox
autoware_traffic_light_classifier
autoware_traffic_light_fine_detector
tier4_perception_launch
tier4_simulator_launch

Many packages are missing this dependency. Could you update them?

colcon graph --packages-above autoware_cuda_utils autoware_tensorrt_common --dot | dot -Tsvg -Grankdir=LR -Gnodesep=0.2 -Granksep=0.3 -Gconcentrate=true -o graph.svg

---

Or rather, a question:
The autoware_probabilistic_occupancy_grid_map contains, say, 2 different nodes that share some logic.
One needs cuda and will not be built if there is no nvcc. The other one does not need cuda so will be built no matter what.
Should we use the meta dependency in this case?

@knzo25 I'd suggest that we should not allow partial dependency for cuda for packages. A package should either depend on it or not.

And the building behavior set in CMakeLists.txt should reflect that.

If a node out of this package needs to be used in a non-cuda environment, it should be separated into another package.

But until that happens, we will treat the entire package as a cuda package.

[knzo25]

@xmfcx
I agree with that and for new nodes I am trying hard to separate implementations without duplicating code (what will happen in the concat vs cuda concat).
For the OGM we really needed to acelerate it so I cuda-fied it. Then we got complains that some people needed to run one version (the one that is not heavy) in environments without cuda so I modified the cmake to allow for that 😢

[knzo25]

LGTM

[xmfcx]

About 6 packages are missing the dependency.

• refactor: add autoware_cuda_dependency_meta #10073 (comment)

I'm slightly conflicted about <exec_depend>ency. I think we can exclude them?

[esteve]

@xmfcx

Many packages are missing this dependency. Could you update them?

I don't see why those packages should depend on autoware_cuda_dependency_meta, the rationale for the package as a I understood it is that if they depend on CUDA directly, then they need to depend on autoware_cuda_dependency_meta. Otherwise, if we're just filtering out packages that depend on autoware_tensorrt_common and autoware_cuda_utils, then I don't see the point of autoware_cuda_dependency_meta itself, when we can just filter out via those.

@xmfcx @mitsudome-r @youtalk Could we clarify this? Thanks.

[xmfcx]

Right, I got confused when you said:

if the package depends on CUDA in one way or another, yes, it must depend on autoware_cuda_dependency_meta, even if some of its components can be built without CUDA.

I thought you've already added some packages that depended on CUDA indirectly. Sorry, for my misunderstanding🙇‍♂️.

I've also stated in the original issue to add only direct dependencies so I'm in line with you 👍

autoware_bytetrack also depends on CUDA to build I guess, at least it is using some CUDA related parameters in its cmakelists file.

[xmfcx]

CMakeLists has tensorrt version variable passed to it, gray area?:

• autoware_bytetrack

Indirectly dependent on CUDA packages:

• autoware_detected_object_validation
• autoware_detection_by_tracker
• autoware_raindrop_cluster_filter
• tier4_perception_launch
• tier4_simulator_launch

The rest are directly dependent and the deps are added within this PR ✅

[esteve]

autoware_bytetrack also depends on CUDA to build I guess, at least it is using some CUDA related parameters in its cmakelists file.

You're right, I missed that one. Basically what I did was to grep -ri find_package(CUDA and add autoware_cuda_dependency_meta as a dependency in package.xml, but forgot to check for TensorRT. I'll add the dependency to autoware_bytetrack