Command Line Interface

Testing Farm provides a CLI tool for easy interaction with Testing Farm.

Use cases

Currently, the CLI targets these main use cases:

  • submitting a test request

  • listing and filtering test requests

  • watch a request for completion

  • get a request completion report

  • easy integration with CI systems via small container image quay.io/testing-farm/cli

  • reservation of a test environment

Installation

See the CLI installation documentation.

First Steps

For the CLI to work, make sure to export the TESTING_FARM_API_TOKEN in your environment. If you do not yet have a token, see onboarding.

Testing of the project is currently done via:

export TESTING_FARM_API_TOKEN=<your-token>

The token is specific only to a single ranch. If you want to use more ranches, you will need a token for each of them.

Requesting Testing

Testing Farm provides a sample tmt plan to try out the CLI tool. This plan runs a single test which by default prints out cat /etc/os-release.

  • 🌍 Public Ranch - run sanity test against Fedora Rawhide

    testing-farm request --compose Fedora-Rawhide --git-url https://gitlab.com/testing-farm/tests --plan /testing-farm/sanity

  • 🎩 Red Hat Ranch

    testing-farm request --compose RHEL-9.2.0-Nightly --git-url https://gitlab.com/testing-farm/tests --plan /testing-farm/sanity

If your test is expected to take a long time, you should set the --timeout parameter so that your test is not terminated prematurely. The default is currently 12 hours. Try your tmt test locally first to get an idea of how long it will take - perhaps using time. The timeout is specified in minutes. For example:

testing-farm request --compose RHEL-9.2.0-Nightly --git-url https://gitlab.com/myproject --plan /tests/bigtest --timeout 360

Multiple key=value Pairs per Single Option

As of CLI v0.0.21, released together with Testing Farm 2024-10.5, options which accept key=value pairs, such as:

  • --kickstart,

  • --context (-c),

  • --environment (-e),

  • --secret (-s),

  • --tmt-environment (-T)

  • --tag (-t)

now support setting multiple key=value pairs per single option. For example:

With testing-farm CLI
$ testing-farm request -e "foo=foo bar=bar"
In Testing Farm API:
...
{
    "variables": {
        "foo": "foo",
        "bar": "bar"
    }
}
...

Breaking change for strings with whitespaces

This change affects how strings containing whitespaces should be quoted.

Until v0.0.20

Wrapping the whole key=value pair in quotes if either key or value contained white spaces.

$ testing-farm request --kickstart "foo=script with whitespace"
After v0.0.20

Wrapping the whole key=value pair in quotes and also nested quotes pair of quotes for the key or value that contains whitespaces.

$ testing-farm request --kickstart "foo='script with whitespace'"
In Testing Farm API

The commands above result in this data being sent to the API:

...
{
    "kickstart": {
        "foo": "script with whitespace"
    }

}
...

Reserving Systems

The testing-farm reserve command allows you to reserve a system in the Testing Farm environment for testing purposes. This is particularly useful for debugging unexpected test failures.

The reservation now works in the public ranch as well. Make sure to use the v0.0.19 or a newer version of the testing-farm CLI.

Reservation requires ssh-agent to be running on the system with at least one key added to it. This is due to seamless login experience to the reserved machine.

  • check if you have ssh-agent running and at least one key added

    ssh-add -L

  • in case there is no ssh-agent running, start it with the following command

    eval $(ssh-agent)

  • If you do not have any ssh keys generated, run the following command to generate one

    ssh-keygen

  • Lastly add the generated key to the ssh-agent

    ssh-add

To reserve a system, use the following command:

testing-farm reserve --compose Fedora-Rawhide

By default, the CLI sets the reservation duration to 30 minutes. You can change this duration using the --duration MINUTES option. For example:

testing-farm reserve --compose Fedora-Rawhide --duration 60

Or by using the extend-reservation MINUTES command within the machine:

extend-reservation 60

The maximum reservation time is 12 hours (720 minutes).

SSH is used for accessing the reserved machine. The CLI will, by default, copy all public keys that match the pattern ~/.ssh/*.pub to the machine. You can specify a different public key using the --ssh-public-key PATH option. Globbing can be used to specify multiple keys. For instance:

testing-farm reserve --compose Fedora-Rawhide --ssh-public-key /path/to/my/keys/*.pub

For the container based installation, the option is unsupported.

Similar to the testing-farm request command, the reserve command provides various options to customize the testing environment. You can view all available options using the --help option:

testing-farm reserve --help

See Test Environment and Hardware Requirements pages to find more information.

If you need to return the machine early, execute the following command within the machine:

return2testingfarm

Listing Requests

The list command provides comprehensive viewing and filtering of Testing Farm requests with multiple output formats and advanced filtering options.

The documentation here is not comprehensive. See the --help for all available options.

testing-farm list --help

Your Requests

Listing your requests requires authentication to identify your requests.

 
testing-farm list

When listing requests, the table output format is used by default.

By default, the listing shows only requests created in the last day. To adjust the age of the requests, you can use the --age and --min-age parameters. For example, to show your requests created in the last 30 days, use the command below.

testing-farm list --age 30d

By default, the listing shows all request states. If you want to view only specific states, use the --state option. You can specify multiple states by passing the --state option multiple times.

testing-farm list --state complete --state running

If you want to view details of a specific request, copy the request ID (the UUID4 string) from the list and pass it to the list command.

When listing a single request, the output format defaults to text to display all request details.

 
testing-farm list --id 1720852c-93ce-46b7-9412-4436a4c24406

By default, secrets are hidden when viewing requests. To also expose the secret values, use the --show-secret option.

It is only possible to view secrets for your own requests.

 
testing-farm list --id 1720852c-93ce-46b7-9412-4436a4c24406 --show-secrets

Your Reservations

To view all your active reservations, use the --reservations option.

testing-farm list --reservations

Reservations have a dedicated output format and it is currently not possible to change it via the --format option.

The reservations expose the SSH username and host to use to connect to the machine in the user@ip column. If the user@ip column shows <not-yet-available>, it means that the reservation is in progress but no machine has been provisioned yet.

Reserving machines in certain pools like beaker can take several hours. Also, Testing Farm might have a large queue, which can affect the time until a machine is reserved.

All Requests

The list command supports viewing all Testing Farm requests via the --all option.

Listing all requests does not require authentication, and by default, requests for all ranches are shown.

To list all requests in the last day, use the command below.

The default age of requests shown is a day. Use the --age option to adjust the age of the shown requests.

 
testing-farm list --all

Requests for a Token ID

To view all requests related to a specific token ID, use the --token-id option.

You can find your token ID in the text output format or in the API request details.

 
testing-farm list --token-id 0cfc00a8-94d7-4408-babc-4d0bc43821ea

Output Formats

Output format cannot be forced when listing reservations. Feel free to file an issue if this problem is limiting you.

The command supports multiple output formats using the --format option:

  • table (default) - Tabular display for easy reading

  • json - Machine-readable JSON format

  • text - Simple text output with complete details of the request

  • yaml - YAML format

testing-farm list --format json
testing-farm list --format yaml
testing-farm list --format text

Cancelling Requests

In some cases, you want to prematurely cancel your request. Use the cancel command and provide the request ID or a string containing it.

$ testing-farm cancel 9baab88b-aca6-4652-ad93-8d954e109a25
$ testing-farm cancel https://api.testing-farm.io/v0.1/requests/a0f18d55-2dd5-466d-b2b8-6bd4a60ca12e
$ testing-farm cancel https://artifacts.dev.testing-farm.io/a0f18d55-2dd5-466d-b2b8-6bd4a60ca12e