Managing test plans
The unit for testing in Testground is a
test plan also referred to as simply a plan. This document explains the layout of test plans directory in the Testground home directory, as well is tooling for managing local plans.Anatomy of a test plan
A test plan is a directory which contains the code to be executed during the test and a
toml-formatted manifest file, which explains the plan to the Testground system.Writing new test plans from scratch is covered elsewhere in this tutorial, for now, just know that a
test plan is a directory with a test plan manifest and that each test plan may have one or more test cases.Testground provides some basic functionality to assist importing and creating of test plans.
During execution, an archive of the plan is sent to the Testground daemon, where it is built and executed. Any code or files inside the plan's directory will be available when the plan is built.
Location of test plans
Test plans reside inside the Testground home directory in a subdirectory called
plans. The location of the Testground home is governed by an environment variable $TESTGROUND_HOME and has some sane defaults if this variable is unset.By default, on unix-like operating systems, this directory is in the user's home directory. Don't worry about creating the Testground home directory in advance; it will be created for you when Testground runs.
1
$ tree $TESTGROUND_HOME
2
​
3
testground
4
├── data
5
│  ├── outputs
6
│  └── work
7
├── plans # <-- This is where plans will go!
8
└── sdks
Copied!
Plan management tooling
Test plans can be managed using regular filesystem utilities. However, the Testground tool does have utilities for managing plans which can help to import and manage plans in a predictable way. For the following sections, I'll demonstrate a few management commands along with some standard utilities to explain what the command does.
Creating new plans
1
# Create the plan
2
$ testground plan create -p myplan
3
​
4
# What happened?
5
$ tree $TESTGROUND_HOME
6
​
7
testground
8
├── data
9
│  ├── outputs
10
│  └── work
11
├── plans
12
│  └── myplan # <-- the root of the plan you just created
13
│  ├── go.mod
14
│  ├── main.go # <-- source code for your your new plan
15
│  └── manifest.toml # <-- manifest for your new plan
16
└── sdks
Copied!
You can modify the behaviour of
plan createusing command-line flags to change the module or add a git remote.Importing existing plans
Importing existing plans requires a
--source flag. The source can be either from the local filesystem or downloaded from a git repository. When importing plans from a local filesystem, a symbolic link is created from the source to the plan directory. When git is used, the plan is imported over any protocol supported by git....from the local filesystem
1
# Import multiple plans from your local filesystem
2
# Changing the name to "myplans" (not required)
3
​
4
$ testground plan import --source /local/plans/dir/ --name myplans
5
​
6
created symlink $TESTGROUND_HOME/plans/myplans -> /local/plans/dir/
7
imported plans:
8
myplans/verify uses-data-network
9
myplans/network ping-pong
10
myplans/benchmarks all
11
myplans/benchmarks startup
12
myplans/benchmarks netinit
13
myplans/benchmarks netlinkshape
14
myplans/benchmarks barrier
15
myplans/benchmarks subtree
16
myplans/placebo ok
17
myplans/placebo abort
18
myplans/placebo metrics
19
myplans/placebo panic
20
myplans/placebo stall
21
myplans/example output
22
myplans/example failure
23
myplans/example panic
24
myplans/example params
25
myplans/example sync
26
myplans/example metrics
27
​
28
# What happened?
29
# a symbolic link has been created to point to the source on my local filesystem
30
​
31
$ ls -l $TESTGROUND_HOME/plans
32
​
33
total 3
34
drwxr-xr-x 1 cory users 60 May 4 15:01 myplan
35
lrwxrwxrwx 1 cory users 56 May 4 15:36 myplans -> /local/plans/dir/
Copied!
...from a git repository
1
# Import multiple plans from the same git repo
2
​
3
$ testground plan import --git --source https://github.com/libp2p/test-plans
4
​
5
Enumerating objects: 54, done
6
Counting objects: 100% (54/54), done.
7
Compressing objects: 100% (41/41), done.
8
Total 54 (delta 16), reused 36 (delta 11), pack-reused 0
9
cloned plan $TESTGROUND_HOME/plans/test-plans -> ssh://[email protected]/libp2p/test-plans
10
imported plans:
11
test-plans/dht find-peers
12
test-plans/dht find-providers
13
test-plans/dht provide-stress
14
test-plans/dht store-get-value
15
test-plans/dht get-closest-peers
16
test-plans/dht bootstrap-network
17
test-plans/dht all
18
​
19
​
20
# What happened?
21
# The repository was cloned with the git remote set to the source.
22
​
23
$ cd $TESTGROUND_HOME/plans/test-plans
24
​
25
$ git remote -v
26
​
27
origin [email protected]:libp2p/test-plans (fetch)
28
origin [email protected]:libp2p/test-plans (push)
Copied!
Listing existing test plans
As you can see from the commands above, we have the ability to create new plans which we will write ourselves or import existing plans or collections of plans. Let's show them all with the
list command:1
# Generate a list of all test plans, along with all test cases in each plan.
2
# These are all the plans imported or created
3
​
4
$ testground plan list --testcases
5
​
6
myplan quickstart
7
myplans/benchmarks all
8
myplans/benchmarks startup
9
myplans/benchmarks netinit
10
myplans/benchmarks netlinkshape
11
myplans/benchmarks barrier
12
myplans/benchmarks subtree
13
test-plans/dht find-peers
14
test-plans/dht find-providers
15
test-plans/dht provide-stress
16
test-plans/dht store-get-value
17
test-plans/dht get-closest-peers
18
test-plans/dht bootstrap-network
19
test-plans/dht all
20
myplans/example output
21
myplans/example failure
22
myplans/example panic
23
myplans/example params
24
myplans/example sync
25
myplans/example metrics
26
myplans/network ping-pong
27
myplans/placebo ok
28
myplans/placebo abort
29
myplans/placebo metrics
30
myplans/placebo panic
31
myplans/placebo stall
32
myplans/verify uses-data-network
Copied!
Removing test plans
Finally, let's end by removing a plan we are no longer interested in:
1
# Examples? Who needs em!
2
​
3
$ testground plan rm -p myplans/example
4
​
5
# Oops! This command is destructive, so it needs a confirmation.
6
​
7
$ testground plan rm -p myplans/example --yes
Copied!
​
Last modified 2yr ago