Managing test plans
The unit for testing in Testground is a test plan also referred to as simply a plan. This document explains the layout of test plans directory in the Testground home directory, as well is tooling for managing local plans.
A test plan is a directory which contains the code to be executed during the test and a toml-formatted manifest file, which explains the plan to the Testground system.
Writing new test plans from scratch is covered elsewhere in this tutorial, for now, just know that a test plan is a directory with a test plan manifest and that each test plan may have one or more test cases.
Testground provides some basic functionality to assist importing and creating of test plans.
During execution, an archive of the plan is sent to the Testground daemon, where it is built and executed. Any code or files inside the plan's directory will be available when the plan is built.
Test plans reside inside the Testground home directory in a subdirectory called plans. The location of the Testground home is governed by an environment variable $TESTGROUND_HOME and has some sane defaults if this variable is unset.
By default, on unix-like operating systems, this directory is in the user's home directory. Don't worry about creating the Testground home directory in advance; it will be created for you when Testground runs.
$ tree $TESTGROUND_HOMEtestground├── data│ ├── outputs│ └── work├── plans # <-- This is where plans will go!└── sdks
Test plans can be managed using regular filesystem utilities. However, the Testground tool does have utilities for managing plans which can help to import and manage plans in a predictable way. For the following sections, I'll demonstrate a few management commands along with some standard utilities to explain what the command does.
# Create the plan$ testground plan create -p myplan# What happened?$ tree $TESTGROUND_HOMEtestground├── data│ ├── outputs│ └── work├── plans│ └── myplan # <-- the root of the plan you just created│ ├── go.mod│ ├── main.go # <-- source code for your your new plan│ └── manifest.toml # <-- manifest for your new plan└── sdks
You can modify the behaviour of plan createusing command-line flags to change the module or add a git remote.
Importing existing plans requires a --source flag. The source can be either from the local filesystem or downloaded from a git repository. When importing plans from a local filesystem, a symbolic link is created from the source to the plan directory. When git is used, the plan is imported over any protocol supported by git.
# Import multiple plans from your local filesystem# Changing the name to "myplans" (not required)$ testground plan import --source /local/plans/dir/ --name myplanscreated symlink $TESTGROUND_HOME/plans/myplans -> /local/plans/dir/imported plans:myplans/verify uses-data-networkmyplans/network ping-pongmyplans/benchmarks allmyplans/benchmarks startupmyplans/benchmarks netinitmyplans/benchmarks netlinkshapemyplans/benchmarks barriermyplans/benchmarks subtreemyplans/placebo okmyplans/placebo abortmyplans/placebo metricsmyplans/placebo panicmyplans/placebo stallmyplans/example outputmyplans/example failuremyplans/example panicmyplans/example paramsmyplans/example syncmyplans/example metrics# What happened?# a symbolic link has been created to point to the source on my local filesystem$ ls -l $TESTGROUND_HOME/planstotal 3drwxr-xr-x 1 cory users 60 May 4 15:01 myplanlrwxrwxrwx 1 cory users 56 May 4 15:36 myplans -> /local/plans/dir/
# Import multiple plans from the same git repo$ testground plan import --git --source https://github.com/libp2p/test-plansEnumerating objects: 54, doneCounting objects: 100% (54/54), done.Compressing objects: 100% (41/41), done.Total 54 (delta 16), reused 36 (delta 11), pack-reused 0cloned plan $TESTGROUND_HOME/plans/test-plans -> ssh://git@github.com/libp2p/test-plansimported plans:test-plans/dht find-peerstest-plans/dht find-providerstest-plans/dht provide-stresstest-plans/dht store-get-valuetest-plans/dht get-closest-peerstest-plans/dht bootstrap-networktest-plans/dht all# What happened?# The repository was cloned with the git remote set to the source.$ cd $TESTGROUND_HOME/plans/test-plans$ git remote -vorigin git@github.com:libp2p/test-plans (fetch)origin git@github.com:libp2p/test-plans (push)
As you can see from the commands above, we have the ability to create new plans which we will write ourselves or import existing plans or collections of plans. Let's show them all with the list command:
# Generate a list of all test plans, along with all test cases in each plan.# These are all the plans imported or created$ testground plan list --testcasesmyplan quickstartmyplans/benchmarks allmyplans/benchmarks startupmyplans/benchmarks netinitmyplans/benchmarks netlinkshapemyplans/benchmarks barriermyplans/benchmarks subtreetest-plans/dht find-peerstest-plans/dht find-providerstest-plans/dht provide-stresstest-plans/dht store-get-valuetest-plans/dht get-closest-peerstest-plans/dht bootstrap-networktest-plans/dht allmyplans/example outputmyplans/example failuremyplans/example panicmyplans/example paramsmyplans/example syncmyplans/example metricsmyplans/network ping-pongmyplans/placebo okmyplans/placebo abortmyplans/placebo metricsmyplans/placebo panicmyplans/placebo stallmyplans/verify uses-data-network
Finally, let's end by removing a plan we are no longer interested in:
# Examples? Who needs em!$ testground plan rm -p myplans/example# Oops! This command is destructive, so it needs a confirmation.$ testground plan rm -p myplans/example --yes
