Improving distribution tests in conformance

[LiorLieberman]

On #3827 (comment)

@guicassolato suggested the below:

Worth adding some entropy to each iteration to mitigate the risk of coordinated execution perhaps? For example: introducing jitter, randomising the payload, or anything else that varies the sample across iterations (without affecting expected weight distribution).

It would be nice to have something like thig

/good-first-issue

[k8s-ci-robot]

@LiorLieberman:
This request has been marked as suitable for new contributors.

Guidelines

Please ensure that the issue body includes answers to the following questions:

• Why are we solving this issue?
• To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points?
• Does this issue have zero to low barrier of entry?
• How can the assignee reach out to you for help?

For more details on the requirements of such an issue, please see here and ensure that they are met.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-good-first-issue command.

In response to this:

On #3827 (comment)

@guicassolato suggested the below:

Worth adding some entropy to each iteration to mitigate the risk of coordinated execution perhaps? For example: introducing jitter, randomising the payload, or anything else that varies the sample across iterations (without affecting expected weight distribution).

It would be nice to have something like thig

/good-first-issue

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[carsontham]

/assign

[chris-kaiser-7]

Can I help with this issue?

[LiorLieberman]

Sure!
/assign @chris-kaiser-7

[carsontham]

hi @LiorLieberman, my apologies for not submitting a PR yet, but I was spending the past few days trying to understand the tests and also running the test locally.

So far, I've ran the tests using the below command, without making any changes ( I wanted to make sure I was running the tests correctly and not break anything ).

go test -v ./conformance \
--run TestConformance/MeshHTTPRouteWeight \
--supported-features=Mesh,HTTPRoute

And the test is returning me failure even without adding any entropy:

    httproute-weight.go:64: Traffic distribution test failed (10/10): backend "echo-v1" weighted traffic of 0.524 not within tolerance 0.7 (+/-0.050000)
        backend "echo-v2" weighted traffic of 0.476 not within tolerance 0.3 (+/-0.050000)
    httproute-weight.go:69: Weighted distribution tests failed
=== NAME  TestConformance/MeshHTTPRouteWeight
    apply.go:283: 2025-06-10T18:05:28.312926+08:00: Deleting mesh-weighted-backends HTTPRoute
=== NAME  TestConformance
    apply.go:283: 2025-06-10T18:05:28.317196+08:00: Deleting echo-v1 Deployment
    apply.go:283: 2025-06-10T18:05:28.320049+08:00: Deleting gateway-conformance-mesh-consumer Namespace
    apply.go:283: 2025-06-10T18:05:28.322049+08:00: Deleting echo Service
    apply.go:283: 2025-06-10T18:05:28.330848+08:00: Deleting echo-v2 Service
    apply.go:283: 2025-06-10T18:05:28.341497+08:00: Deleting echo-v2 Deployment
    apply.go:283: 2025-06-10T18:05:28.345876+08:00: Deleting echo-v1 Service
    apply.go:283: 2025-06-10T18:05:28.352877+08:00: Deleting echo-v1 Deployment
    apply.go:283: 2025-06-10T18:05:28.357337+08:00: Deleting gateway-conformance-mesh Namespace
--- FAIL: TestConformance (73.35s)
    --- FAIL: TestConformance/MeshHTTPRouteWeight (71.08s)
        --- FAIL: TestConformance/MeshHTTPRouteWeight/Requests_should_have_a_distribution_that_matches_the_weight (71.06s)

I'm not sure if I'm running the tests wrongly, but could you point me to the right direction? thanks !

[LiorLieberman]

@carsontham Sorry! that was last night my time and I did not see that you already assinged.

Please go ahead, open a PR and I can help you debug it on the PR.

@chris-kaiser-7 - I recommend waiting with this one since @carsontham is already working on it.

/assign @carsontham

[carsontham]

hey @LiorLieberman no worries at all! will come up with a PR soon :) Thanks alot!