Skip to content

Run unit and e2e tests in parallel #19161

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

nubtron
Copy link
Contributor

@nubtron nubtron commented Nov 28, 2024

WIP

What does this PR do?

Motivation

Review checklist (to be filled by reviewers)

  • Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
  • Add the qa/skip-qa label if the PR doesn't need to be tested during QA.
  • If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

Copy link

codecov bot commented Nov 28, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 88.11%. Comparing base (2501d5a) to head (5806560).
Report is 870 commits behind head on master.

Additional details and impacted files
Flag Coverage Δ
active_directory 86.36% <ø> (ø)
activemq 52.80% <ø> (ø)
activemq_xml 82.20% <ø> (ø)
aerospike 87.04% <ø> (+16.44%) ⬆️
airflow 92.64% <ø> (ø)
amazon_msk 89.19% <ø> (ø)
ambari 85.75% <ø> (ø)
apache 95.08% <ø> (ø)
appgate_sdp 93.93% <ø> (ø)
arangodb 98.23% <ø> (ø)
argo_rollouts 90.00% <ø> (ø)
argo_workflows 87.87% <ø> (ø)
argocd 87.23% <ø> (ø)
aspdotnet 100.00% <ø> (ø)
avi_vantage 91.35% <ø> (ø)
aws_neuron 92.42% <ø> (ø)
azure_iot_edge 82.08% <ø> (ø)
boundary 100.00% <ø> (ø)
btrfs 83.33% <ø> (ø)
cacti 87.90% <ø> (∅)
calico 84.61% <ø> (ø)
cassandra 66.66% <ø> (ø)
cassandra_nodetool 93.16% <ø> (ø)
ceph 91.04% <ø> (ø)
cert_manager 77.41% <ø> (∅)
cilium 77.92% <ø> (ø)
cisco_aci 89.59% <ø> (ø)
citrix_hypervisor 87.50% <ø> (ø)
clickhouse 94.55% <ø> (+0.02%) ⬆️
cloud_foundry_api 96.11% <ø> (ø)
cloudera 99.51% <ø> (ø)
cockroachdb 92.98% <ø> (ø)
confluent_platform 81.96% <ø> (?)
consul 91.92% <ø> (∅)
coredns 95.65% <ø> (ø)
couch 95.17% <ø> (∅)
couchbase 84.87% <ø> (ø)
crio 89.79% <ø> (ø)
datadog_checks_base 89.29% <ø> (∅)
datadog_checks_dev 77.63% <ø> (∅)
datadog_checks_downloader 78.12% <ø> (ø)
datadog_cluster_agent 90.19% <ø> (ø)
dcgm 93.54% <ø> (ø)
ddev 88.03% <ø> (∅)
directory 96.46% <ø> (ø)
disk 90.86% <ø> (+1.42%) ⬆️
dns_check 93.81% <ø> (ø)
dotnetclr 80.64% <ø> (ø)
druid 97.70% <ø> (ø)
ecs_fargate 83.71% <ø> (ø)
eks_fargate 94.05% <ø> (ø)
envoy 89.95% <ø> (∅)
esxi 93.92% <ø> (∅)
etcd 95.56% <ø> (∅)
exchange_server 88.09% <ø> (ø)
external_dns 89.28% <ø> (ø)
fluentd 84.21% <ø> (ø)
fluxcd 88.31% <ø> (ø)
fly_io 97.13% <ø> (ø)
foundationdb 83.83% <ø> (∅)
gearmand 77.84% <ø> (ø)
gitlab_runner 92.76% <ø> (∅)
glusterfs 80.00% <ø> (∅)
go_expvar 92.66% <ø> (ø)
gunicorn 92.16% <ø> (∅)
haproxy ∅ <ø> (∅)
harbor 89.65% <ø> (ø)
hazelcast 92.30% <ø> (∅)
hdfs_datanode 89.63% <ø> (∅)
hdfs_namenode 86.60% <ø> (∅)
hive 51.42% <ø> (ø)
hivemq 61.90% <ø> (ø)
http_check 94.25% <ø> (ø)
hudi 73.91% <ø> (∅)
ibm_ace ∅ <ø> (∅)
ibm_db2 86.29% <ø> (∅)
ibm_i 82.36% <ø> (ø)
ibm_mq ∅ <ø> (∅)
ibm_was 96.06% <ø> (ø)
ignite 46.66% <ø> (∅)
iis 93.96% <ø> (+0.08%) ⬆️
impala 97.97% <ø> (∅)
istio 77.86% <ø> (ø)
jboss_wildfly 47.36% <ø> (ø)
kafka 64.70% <ø> (ø)
karpenter 95.06% <ø> (ø)
kong 88.42% <ø> (∅)
kube_apiserver_metrics 97.75% <ø> (ø)
kube_controller_manager 97.88% <ø> (ø)
kube_dns 95.94% <ø> (ø)
kube_metrics_server 94.87% <ø> (ø)
kube_proxy 96.80% <ø> (ø)
kube_scheduler 97.92% <ø> (ø)
kubeflow 93.22% <ø> (ø)
kubelet 91.09% <ø> (ø)
kubernetes_cluster_autoscaler 93.22% <ø> (ø)
kubernetes_state 89.49% <ø> (ø)
kubevirt_api 82.75% <ø> (∅)
kubevirt_controller 85.36% <ø> (∅)
kubevirt_handler 91.32% <ø> (∅)
kyototycoon 85.96% <ø> (ø)
kyverno 82.27% <ø> (-17.73%) ⬇️
lighttpd 83.64% <ø> (ø)
linkerd 84.70% <ø> (ø)
linux_proc_extras 96.20% <ø> (ø)
mapr 82.70% <ø> (ø)
mapreduce 81.99% <ø> (∅)
marathon 83.06% <ø> (ø)
mcache 93.48% <ø> (ø)
mesos_master 89.71% <ø> (ø)
mesos_slave 93.56% <ø> (ø)
nagios 89.01% <ø> (ø)
network ∅ <ø> (∅)
nfsstat 95.20% <ø> (ø)
nginx ∅ <ø> (∅)
nginx_ingress_controller 98.55% <ø> (ø)
nvidia_nim 100.00% <ø> (+6.89%) ⬆️
nvidia_triton 88.52% <ø> (ø)
openldap ∅ <ø> (∅)
openmetrics ∅ <ø> (∅)
openstack ∅ <ø> (∅)
pdh_check ∅ <ø> (∅)
pgbouncer ∅ <ø> (∅)
php_fpm ∅ <ø> (∅)
postfix ∅ <ø> (∅)
powerdns_recursor ∅ <ø> (∅)
presto ∅ <ø> (∅)
process ∅ <ø> (∅)
prometheus ∅ <ø> (∅)
proxysql ∅ <ø> (∅)
pulsar ∅ <ø> (∅)
rethinkdb ∅ <ø> (∅)
riak ∅ <ø> (∅)
riakcs ∅ <ø> (∅)
silk ∅ <ø> (∅)
singlestore ∅ <ø> (∅)
slurm ∅ <ø> (∅)
snowflake ∅ <ø> (∅)
solr ?
spark 93.47% <ø> (∅)
strimzi ∅ <ø> (∅)
tcp_check ∅ <ø> (∅)
windows_performance_counters ∅ <ø> (∅)
wmi_check ∅ <ø> (∅)

Flags with carried forward coverage won't be shown. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link

Test Results

0 files   -    663  0 suites   - 663   0s ⏱️ - 4h 53m 39s
0 tests  -  7 475  0 ✅  -  7 228  0 💤  -    76  0 ❌  - 171 
0 runs   - 26 994  0 ✅  - 23 150  0 💤  - 3 673  0 ❌  - 171 

Results for commit 5806560. ± Comparison against base commit 2501d5a.

Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR updates the CI configuration to run unit and end-to-end tests in parallel by introducing a strategy matrix in the workflow. Key changes include:

  • Adding a matrix strategy with a test_type parameter to distinguish unit and e2e jobs.
  • Appending the matrix test type to environment variables and artifact names.
  • Adjusting job conditions accordingly and updating the master workflow to trigger on pull requests to master.

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
.github/workflows/test-target.yml Added a strategy matrix and modified steps to run tests in parallel.
.github/workflows/master.yml Added pull_request trigger on master to support the new testing flow.
Comments suppressed due to low confidence (2)

.github/workflows/master.yml:7

  • Confirm that adding a pull_request trigger for the master branch is intentional, as this change may lead to additional workflow runs which could affect CI utilization.
pull_request:

.github/workflows/test-target.yml:96

  • Ensure that both unit and e2e tests have appropriate test coverage and that downstream steps correctly aggregate their respective artifacts based on the matrix test type.
matrix:
        test_type: ['unit', 'e2e']

@dd-devflow dd-devflow bot closed this May 31, 2025
@dd-devflow dd-devflow bot deleted the nubtron/more-test-concurrency branch May 31, 2025 00:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant