Adding release test capability #352
Conversation
* Adding initial test release file * Format fix * Format fix * It appears ordering matters * It appears ordering matters * ok if statement not a fan there * ok if statement not a fan there * Adding matrix generation * Fixing permissions * adding python * Correcting json * simplifying matrix * Fixing tar * Adding mkdir * Adding release notes * testing with ubuntu * Testing this * Missing quotes * Trying this * Trying this * Writing to markdown * Okay this should work * Adding cancel step * markdown cleanup * Adding cleanup and docs
| - name: Run hipBLASLt tests | ||
| if: '!cancelled()' | ||
| id: hipblaslt-tests | ||
| run: | | ||
| source ${VENV_DIR}/bin/activate | ||
| ${THEROCK_BIN_DIR}/hipblaslt-test --gtest_filter=*pre_checkin* | ||
|
|
||
| - name: Run rocBLAS tests | ||
| if: '!cancelled()' | ||
| id: rocblas-tests | ||
| run: | | ||
| ROCBLAS_TENSILE_LIBPATH="${THEROCK_BIN_DIR}/lib/rocblas/library/" | ||
| source ${VENV_DIR}/bin/activate | ||
| ${THEROCK_BIN_DIR}/rocblas-test --yaml ${THEROCK_BIN_DIR}/rocblas_smoke.yaml |
There was a problem hiding this comment.
Can we reuse here what you've recently added to the CI?
There was a problem hiding this comment.
i'll need to do some rework on both sides to consolidate and share tests, since it seems duplicating
and for these tests, it'll involve some upload/download assets and hopefully that won't be too costly in time
There was a problem hiding this comment.
i'll actually scope this PR to just adding the capability to test, then in a future PR, i'll work on consolidating and combining the test efforts
geomin12
added
gfx94X-linux
bug
| type: string | ||
|
|
||
| jobs: | ||
| generate_release_matrix: |
There was a problem hiding this comment.
I am not sure the job need to generate the release matrix to test for itself. This could be an input form the prior job. With this it's also more easy to trigger exactly one run on a specific system and for a specific GPU.
There was a problem hiding this comment.
Looks overall pretty good! The one thing I am little concerned about is if appending to a release like https://github.com/ROCm/TheRock/releases/tag/nightly-release won't be a little noisy, in particular as soon as further components will be tested. I think we need to improve this somehow ...
yes good point, i also had concerns since that data will be appending essentially forever My current thought is, we can append the results in each "job github output" like this iree example, then in the release notes, we can just add SUCCESS/FAIL notes on the most recent run. That way, we can keep track of historic results but not make the |
Oh yeah, I like that 👍 |
This file gets triggered for any nightly release workflow finishing, but it's very niche to mi300 since that is the only machine we can run tests on.
Changes:
Tested results and run with ubuntu-latest (obviously causes failure) in this release notes page and here is the workflow run
This will enable testing for nightly releases and report results
Next todos:
test_linux_packages.yml... opportunity to combine 🤑!gfx94XCloses #368