f71277149a
* ci: adding scale pipeline for cilium * change timeout * change timeout * update net policies * use acn build pool to avoid azp delays * address comments -- update build pool var and remove timeouts from scale * address comments -- set node/pod counts and test input as variables * remove all test resources * check apachebench rollout status * collect more cpu/mem results * add cns restart and fix artifact upload * add cns restart and fix artifact upload * update name for artifact publishing * update apachebench artifact collection * update apachebench artifact collection * test cns version check * change cns update * update artifact directory name * add netperf testing stage * give permissions to netperf script * change netperf steps * update path to netperf yaml * change netperf deployment to 2 * get correct pods in netperf script * publish netperf results * publish netperf results * add same vm test for netperf * netperf script to print pod values * netperf find nodes logic * netperf find nodes logic |
||
|---|---|---|
| .. | ||
| config | ||
| connectivity | ||
| templates | ||
| utils | ||
| README.md | ||
| install-kwok.sh | ||
| label-nodes.sh | ||
| run-kwok.sh | ||
| test-scale.sh | ||
README.md
Overview
Scripts for scale testing our components in AKS with fake and/or real resources.
Fake Resources
Scripts can use KWOK to simulate running Pods. KWOK can instantly run thousands of fake VMs and Pods.
This saves us from:
- Large resource costs.
- Hours waiting for VMs and Pods to bootup.
Usage
- Create AKS cluster with
--uptime-slaand create any nodepools. - If making KWOK Pods, run
run-kwok.shin the background. - Scale with
test-scale.sh. Specify number of Deployments, Pod replicas, NetworkPolicies, and labels for Pods. Can also delete/re-add objects to cause churn. - Test connectivity with
connectivity/test-connectivity.sh.
Example Runs
./test-scale.sh --max-kwok-pods-per-node=50 \
--num-kwok-deployments=10 \
--num-kwok-replicas=1 \
--max-real-pods-per-node=30 \
--num-real-deployments=5 \
--num-real-services=4 \
--num-real-replicas=2 \
--num-network-policies=1 \
--num-unapplied-network-policies=10 \
--num-unique-labels-per-pod=2 \
--num-unique-labels-per-deployment=2 \
--num-shared-labels-per-pod=10 \
--delete-labels \
--delete-labels-interval=30 \
--delete-labels-times=2 \
--delete-netpols \
--delete-netpols-interval=0 \
--delete-netpols-times=1 \
--delete-kwok-pods=10 \
--delete-real-pods=5 \
--delete-pods-interval=120 \
--delete-pods-times=2
Note: you must run ./test-scale.sh first with --num-network-policies=1 or more, and --num-shared-labels-per-pod=3 or more.
./test-connectivity.sh --num-scale-pods-to-verify=all \
--max-wait-for-initial-connectivity=600 \
--max-wait-after-adding-netpol=120