cluster:k8s runner of Testground enables you to test distributed/p2p systems at scale.
cluster:k8s Testground runner is capable of launching test workloads comprising 10k+ instances, and we aim to reach 100k at some point.
The IPFS and libp2p projects have used these scripts and playbooks to deploy large-scale test infrastructure. By crafting test scenarios that exercise components at such scale, we have been able to run simulations, carry out attacks, perform benchmarks, and execute all kinds of tests to validate correctness and performance.
Kubernetes Operations (
kops) is a tool which helps to create, destroy, upgrade and maintain production-grade Kubernetes clusters from the command line. We use it to create a Kubernetes cluster on AWS.
We use CoreOS Flannel for networking on Kubernetes - for the default Kubernetes network, which in Testground terms is called the
kops uses 100.96.0.0/11 for pod CIDR range, so this is what we use for the
We use Weave Net for the
data plane on Testground - a secondary overlay network that we attach containers to on-demand.
We configure Weave to use 18.104.22.168/4 as CIDR (we want to test
libp2p nodes with IPs in public ranges), so this is the CIDR for the Testground
data network. The
sidecar component is responsible for setting up the
data network for every testplan instance.
In order to have two different networks attached to pods in Kubernetes, we run the CNI-Genie CNI.