Skip to content

refactor(autoware_lidar_transfusion): split config #8205

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

amadeuszsz
Copy link
Contributor

Description

This PR split autoware_lidar_transfusion parameter file for two configs:

  • Parameters strictly related to deployed model.
  • Parameters related to runtime behavior.

The same approach is applied in autoware_lidar_centerpoint package.

Related links

Parent Issue:

How was this PR tested?

Notes for reviewers

None.

Interface changes

None.

Effects on system behavior

None.

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>
@github-actions github-actions bot added component:perception Advanced sensor data processing and environment understanding. (auto-assigned) component:launch Launch files, scripts and initialization tools. (auto-assigned) labels Jul 26, 2024
Copy link

github-actions bot commented Jul 26, 2024

Thank you for contributing to the Autoware project!

🚧 If your pull request is in progress, switch it to draft mode.

Please ensure:

@amadeuszsz amadeuszsz marked this pull request as ready for review July 29, 2024 02:20
@amadeuszsz amadeuszsz self-assigned this Jul 29, 2024
@amadeuszsz amadeuszsz added the run:build-and-test-differential Mark to enable build-and-test-differential workflow. (used-by-ci) label Jul 29, 2024
Copy link

codecov bot commented Jul 29, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 29.20%. Comparing base (5093495) to head (13e3fcd).
Report is 10 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8205      +/-   ##
==========================================
- Coverage   29.24%   29.20%   -0.04%     
==========================================
  Files        1600     1601       +1     
  Lines      117738   117551     -187     
  Branches    50720    50654      -66     
==========================================
- Hits        34427    34335      -92     
+ Misses      74120    74004     -116     
- Partials     9191     9212      +21     
Flag Coverage Δ *Carryforward flag
differential 4.91% <ø> (?)
total 29.20% <ø> (-0.04%) ⬇️ Carriedforward from 9c3f1f2

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@amadeuszsz
Copy link
Contributor Author

@kminoda @knzo25
json-schema-check fails due to same file name prefix here.
Unfortunately, we can't remove asterisk from {base_name}*.param.yaml in CI because this convention is widely used among Autoware packages to apply schema file on multiple param files. We can:

  • Change file name convention from <MODEL>_ml_package.param.yaml to ml_package_<MODEL>.param.yaml. We will need to change deployment path and launch files for transfusion, centerpoint and pointpainting.
  • Bypass this test by file extension thus don't check this config in CI as it is now in centerpoint.

I would go for change naming convention if I didn't miss any drawback. If you have no objection, I will apply this for centerpoint and pointpainting as well.

@kminoda
Copy link
Contributor

kminoda commented Jul 29, 2024

@amadeuszsz Thanks for the proposal 🙏
However, I don't think it is a good idea to rename the parameter file name, as they are already widely used both in Autoware and Pilot.Auto / Web.Auto, and changing those file names would take a bit of time.

So I would say it's better to consider a workaround.

@github-actions github-actions bot added the type:documentation Creating or refining documentation. (auto-assigned) label Jul 30, 2024
@amadeuszsz amadeuszsz added the run:deploy-docs Mark for deploy-docs action generation. (used-by-ci) label Jul 30, 2024
Copy link

@knzo25
Copy link
Contributor

knzo25 commented Aug 1, 2024

@amadeuszsz
In centerpoint we face similar concerns, and as a workaround (#7421) we are considering calling the vanilla model centerpoint_base, which sounds awful, but is compatible with the schema.
I agree with your name change, but also understand that the change would affect several places, so it would time...

@amadeuszsz
Copy link
Contributor Author

@amadeuszsz In centerpoint we face similar concerns, and as a workaround (#7421) we are considering calling the vanilla model centerpoint_base, which sounds awful, but is compatible with the schema. I agree with your name change, but also understand that the change would affect several places, so it would time...

I'm not a big fan of this. It will require few copies of same schema files and still affects many places due to file renaming, as you said.

We could restructure design for both packages to:

|-config
|----|-ml_package.param.yaml
|----|-ml_package_tiny.param.yaml
|----|-ml_package_sigma.param.yaml
|----|-centerpoint.param.yaml
|----|-centerpoint_tiny.param.yaml
|----|-centerpoint_sigma.param.yaml
|----|-detection_class_remapper.param.yaml
|
|-schema
|----|-ml_package.schema.json
|----|-centerpoint.schema.json
|----|-detection_class_remapper.schema.json

At the same time we can keep old configs on cloud server to let people pull files if they didn't update repository.

@knzo25
Copy link
Contributor

knzo25 commented Sep 2, 2024

Update: we decided to skip the problematic schemas in the meantime, so this PR can be merged

Copy link
Contributor

@knzo25 knzo25 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@YoshiRi YoshiRi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@amadeuszsz amadeuszsz merged commit 631d967 into autowarefoundation:main Sep 3, 2024
25 checks passed
ktro2828 pushed a commit to ktro2828/autoware.universe that referenced this pull request Sep 18, 2024
…n#8205)

* refactor(autoware_lidar_transfusion): split config

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>

* style(pre-commit): autofix

* chore(autoware_lidar_transfusion): bypass schema CI workflow

Signed-off-by: amadeuszsz <amadeusz.szymko.2@tier4.jp>

---------

Signed-off-by: Amadeusz Szymko <amadeusz.szymko.2@tier4.jp>
Signed-off-by: amadeuszsz <amadeusz.szymko.2@tier4.jp>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Kenzo Lobos Tsunekawa <kenzo.lobos@tier4.jp>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component:launch Launch files, scripts and initialization tools. (auto-assigned) component:perception Advanced sensor data processing and environment understanding. (auto-assigned) run:build-and-test-differential Mark to enable build-and-test-differential workflow. (used-by-ci) run:deploy-docs Mark for deploy-docs action generation. (used-by-ci) tag:require-cuda-build-and-test type:documentation Creating or refining documentation. (auto-assigned)
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

4 participants