v0.5.3
Runner library
BUILDER LIBRARY
Troubleshooting
Testground is still in early stage of development, so it is possible that:
- Testground crashes
- One of the underlying systems that Testground uses crashes (
Kubernetes,weave,redis, etc.) - Testground doesn't properly clean-up after a test run
- etc.
Here are a few commands that could be helpful for you to inspect the state of your Kubernetes cluster and clean up after Testground:
Delete all pods related to a test plan
Delete all pods that have the
testground.plan=dht label. This is useful in case you used the --run-cfg keep_service=true setting on Testground.1
$ kubectl delete pods -l testground.plan=dht --grace-period=0 --force
Copied!
Restart the sidecar
Restart the
sidecar daemon which manages networks for all testplans1
$ kubectl delete pods -l name=testground-sidecar --grace-period=0 --force
Copied!
Review running, completed, failed pods
You can check all running pods with
1
$ kubectl get pods -o wide
Copied!
Another useful combination is
watching for pods that are not in Running state or that are failing their health checks, with:1
# watch all non-running pods
2
watch 'kubectl get pods --all-namespaces -o wide | grep -v Running'
3
​
4
# watch all not-ready pods
5
watch 'kubectl get pods --all-namespaces -o wide | grep "0\/1"'
Copied!
Get logs from a given pod
1
$ kubectl logs <pod-id, e.g. tg-dht-c95b5>
Copied!
Get access to the Redis shell
1
$ kubectl port-forward svc/testground-infra-redis-master 6379:6379 &
2
$ redis-cli -h localhost -p 6379
Copied!