Improve distribution tests in conformance for MeshHTTPRouteWeight #3855

carsontham · 2025-06-12T17:36:38Z

What type of PR is this?

/area conformance-test

What this PR does / why we need it:
Adding entropy to distribution tests as per comment #3827 (comment)

This is done by introducing random jitter by adding either a short delay, or adding random header values, or both.

Which issue(s) this PR fixes:

Does this PR introduce a user-facing change?:

Improve mesh conformance distribution tests for httproute weights routing

Signed-off-by: carsontham <carsontham@outlook.com>

k8s-ci-robot · 2025-06-12T17:36:40Z

@carsontham: The label(s) kind/conformance-test cannot be applied, because the repository doesn't have them.

In response to this:

What type of PR is this?

/kind conformance-test

What this PR does / why we need it:
Adding entropy to distribution tests to mitigate the risk of coordinated execution.

This is done by introducing random jitter by adding either a short delay, or adding random header values, or both.

Which issue(s) this PR fixes:

Fixes #3832

Does this PR introduce a user-facing change?:
Improve mesh conformance distribution tests for httproute weights routing

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot · 2025-06-12T17:36:46Z

Welcome @carsontham!

It looks like this is your first PR to kubernetes-sigs/gateway-api 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/gateway-api has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

k8s-ci-robot · 2025-06-12T17:36:47Z

Hi @carsontham. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

carsontham · 2025-06-12T17:37:57Z

/area conformance-test

mikemorris · 2025-06-12T21:51:58Z

Nice, thanks for helping out with this!

/ok-to-test

LiorLieberman · 2025-06-13T00:21:17Z

Thanks @carsontham ! did you try to run conformance and see that it works?

carsontham · 2025-06-13T07:48:10Z

Hey @LiorLieberman ! I tried running the conformance test by doing something like this:

go test -v ./conformance \
--run TestConformance/MeshHTTPRouteWeight \
--supported-features=Mesh,HTTPRoute

and the results are like:

 httproute-weight.go:64: Traffic distribution test failed (10/10): backend "echo-v1" weighted traffic of 0.474 not within tolerance 0.7 (+/-0.050000)
        backend "echo-v2" weighted traffic of 0.526 not within tolerance 0.3 (+/-0.050000)
    httproute-weight.go:69: Weighted distribution tests failed
=== NAME  TestConformance/MeshHTTPRouteWeight
    apply.go:283: 2025-06-13T15:41:15.173082+08:00: Deleting mesh-weighted-backends HTTPRoute

I also tried running the MeshHTTPRouteWeight test before making this PR, and observed similar results. I'm not too sure if it is because I am running the tests wrongly, or if the implementations does not conform to the weighted backend yet. Could you point me in the correct direction so that I can run the test properly? thanks!

LiorLieberman · 2025-06-13T16:17:18Z

Hey @LiorLieberman ! I tried running the conformance test by doing something like this:
go test -v ./conformance \
--run TestConformance/MeshHTTPRouteWeight \
--supported-features=Mesh,HTTPRoute
and the results are like:
 httproute-weight.go:64: Traffic distribution test failed (10/10): backend "echo-v1" weighted traffic of 0.474 not within tolerance 0.7 (+/-0.050000)
        backend "echo-v2" weighted traffic of 0.526 not within tolerance 0.3 (+/-0.050000)
    httproute-weight.go:69: Weighted distribution tests failed
=== NAME  TestConformance/MeshHTTPRouteWeight
    apply.go:283: 2025-06-13T15:41:15.173082+08:00: Deleting mesh-weighted-backends HTTPRoute
I also tried running the MeshHTTPRouteWeight test before making this PR, and observed similar results. I'm not too sure if it is because I am running the tests wrongly, or if the implementations does not conform to the weighted backend yet. Could you point me in the correct direction so that I can run the test properly? thanks!

Does #3826 (comment) help? Install istio and follow the command I listed here.

carsontham · 2025-06-13T18:25:40Z

hi @LiorLieberman ! I managed to get the test running using the command as advised. Thanks!

=== NAME  TestConformance/MeshHTTPRouteWeight
    apply.go:283: 2025-06-14T02:24:09.972321+08:00: Deleting mesh-weighted-backends HTTPRoute
--- PASS: TestConformance (20.08s)
    --- PASS: TestConformance/MeshHTTPRouteWeight (19.82s)
        --- PASS: TestConformance/MeshHTTPRouteWeight/Requests_should_have_a_distribution_that_matches_the_weight (19.71s)
PASS
ok  	sigs.k8s.io/gateway-api/conformance	21.450s

LiorLieberman · 2025-06-14T00:15:02Z

Thanks! did you run with. -v? would like to see the whole output

…rites

carsontham · 2025-06-14T03:41:34Z

Thanks! did you run with. -v? would like to see the whole output

I've ran the command here:

go test -v ./conformance --gateway-class istio --run TestConformance/MeshHTTPRouteWeight --cleanup-base-resources=false  --supported-features=Mesh,HTTPRoute -namespace-labels istio-injection=enabled

attaching the test logs as file as there is character limits for comments on Github.
MeshHTTPRouteWeight.log

Thanks !

LiorLieberman

Thanks @carsontham! this looks good to me with one comment.

Do you also want to add it to conformance/tests/httproute-weight.go and perhaps make the function in a shared package somewhere so you dont need to duplicate the code between the files?

/approve

/cc @guicassolato for addtl review

LiorLieberman · 2025-06-15T02:21:31Z

conformance/tests/mesh/httproute-weight.go

+// addEntropy adds jitter to the request by adding either a delay up to 1 second, or a random header value, or both.
+func addEntropy(exp *http.ExpectedResponse) {
+	delay := func() { time.Sleep(time.Duration(rand.Intn(1000)) * time.Millisecond) } //nolint:gosec // This is test code to get random value.
+	randomHeader := func() {


why do we need this no lint?

I noticed that earlier before adding the no lint, prow test was failing here due to:
G404: Use of weak random number generator (math/rand or math/rand/v2 instead of crypto/rand) (gosec)

so i followed a similar example here, to ignore linter for testing purposes.

maybe using crypto.rand would make it happy?

fixed! slightly lengthier due to error handling 😃

also re-ran the tests after using crypto.rand, attaching the new output test logs here:
MeshHTTPRouteWeight_updated.log

k8s-ci-robot · 2025-06-15T02:23:57Z

@LiorLieberman: GitHub didn't allow me to request PR reviews from the following users: for, review.

Note that only kubernetes-sigs members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

Thanks @carsontham! this looks good to me with one comment.

Do you also want to add it to conformance/tests/httproute-weight.go and perhaps make the function in a shared package somewhere so you dont need to duplicate the code between the files?

/approve

/cc @guicassolato for addtl review

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

k8s-ci-robot · 2025-06-15T02:24:00Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: carsontham, LiorLieberman

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~conformance/OWNERS~~ [LiorLieberman]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

carsontham · 2025-06-15T06:09:10Z

Do you also want to add it to conformance/tests/httproute-weight.go and perhaps make the function in a shared package somewhere so you dont need to duplicate the code between the files?

Hey @LiorLieberman! sure thing, but taking a quick look, not too sure where i could put this shared function, perhaps somewhere like conformance/utils/http/http.go ?

also do you prefer it to be in the same current PR? or should I create another PR after this gets approved?

LiorLieberman · 2025-06-15T06:31:43Z

I was thinking somewhere here maybe? https://github.com/kubernetes-sigs/gateway-api/tree/main/conformance/utils.

You could do another PR yeah, and tag me there so its easier review.

once we finish the small comment thread we have I can lgtm this

carsontham · 2025-06-15T08:28:18Z

I was thinking somewhere here maybe? https://github.com/kubernetes-sigs/gateway-api/tree/main/conformance/utils.
You could do another PR yeah, and tag me there so its easier review.
once we finish the small comment thread we have I can lgtm this

sure! i will follow up with a continuation PR after this PR is done. thanks alot @LiorLieberman!

LiorLieberman · 2025-06-16T00:10:10Z

Sweet, thanks @carsontham!
Please open another PR for the rest.

/lgtm
/hold for @guicassolato to review.

added entropy to mesh weight test

65f128d

Signed-off-by: carsontham <carsontham@outlook.com>

k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Jun 12, 2025

k8s-ci-robot requested review from kflynn and mikemorris June 12, 2025 17:36

k8s-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jun 12, 2025

k8s-ci-robot added the area/conformance-test Issues or PRs related to Conformance tests. label Jun 12, 2025

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jun 12, 2025

added nolint:gosec to skip linter issues for conformance test

f056dd5

move creation of header map to entropy func to avoid concurrent map w…

812f3db

…rites

LiorLieberman reviewed Jun 15, 2025

View reviewed changes

k8s-ci-robot requested a review from guicassolato June 15, 2025 02:23

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 15, 2025

use crypto/rand instead of math/rand

3097960

k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Jun 15, 2025

carsontham added 2 commits June 15, 2025 16:45

handle error checking and add default in switch

a0428a1

return error in default for switch case

d524e19

k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 16, 2025

k8s-ci-robot assigned LiorLieberman Jun 16, 2025

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 16, 2025

Improve distribution tests in conformance for MeshHTTPRouteWeight #3855

Are you sure you want to change the base?

Improve distribution tests in conformance for MeshHTTPRouteWeight #3855

Uh oh!

Conversation

carsontham commented Jun 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

k8s-ci-robot commented Jun 12, 2025

Uh oh!

k8s-ci-robot commented Jun 12, 2025

Uh oh!

k8s-ci-robot commented Jun 12, 2025

Uh oh!

carsontham commented Jun 12, 2025

Uh oh!

mikemorris commented Jun 12, 2025

Uh oh!

LiorLieberman commented Jun 13, 2025

Uh oh!

carsontham commented Jun 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

LiorLieberman commented Jun 13, 2025

Uh oh!

carsontham commented Jun 13, 2025

Uh oh!

LiorLieberman commented Jun 14, 2025

Uh oh!

carsontham commented Jun 14, 2025

Uh oh!

LiorLieberman left a comment

Choose a reason for hiding this comment

Uh oh!

LiorLieberman Jun 15, 2025

Choose a reason for hiding this comment

Uh oh!

carsontham Jun 15, 2025

Choose a reason for hiding this comment

Uh oh!

LiorLieberman Jun 15, 2025

Choose a reason for hiding this comment

Uh oh!

carsontham Jun 15, 2025

Choose a reason for hiding this comment

Uh oh!

k8s-ci-robot commented Jun 15, 2025

Uh oh!

k8s-ci-robot commented Jun 15, 2025

Uh oh!

carsontham commented Jun 15, 2025

Uh oh!

LiorLieberman commented Jun 15, 2025

Uh oh!

carsontham commented Jun 15, 2025

Uh oh!

LiorLieberman commented Jun 16, 2025

Uh oh!

Uh oh!

carsontham commented Jun 12, 2025 •

edited

Loading

carsontham commented Jun 13, 2025 •

edited

Loading