RFD2 - Testing Farm Repository Configuration

Status: 💡 planned

This RFD introduces a new way user can adjust a Testing Farm request via a configuration file directly in the git repository.

Introduction

Currently, the API request is the only input to the request setup users can provide to Testing Farm. We encountered various use cases, where this does not provide enough flexibility. These use cases relate to the fact, that the end users of the requests, i.e. the ones consuming the results, are not the ones submitting the request. The requests are submitted for them by other services, like Packit, Fedora CI, Zuul CI etc. By introducing a new way to tune the request, users will gain a lot of flexibility.

Moreover, with the configuration containing also secret values, we can support secrets. The secrets support should make users of Packit happy, as it is the only feature they are not able to provide from our feature set. Making it easy for users to provide safely secrets to the configuration file, we will solve all these use cases.

See section Links for a list of similar concepts in other CI systems and services.

Configuration

Configuration will be stored in a single file .testing-farm.yaml in the root of the project repository. The configuration will extend the Testing Farm request, and will be limited to certain use cases.

The configuration file will have these sections:

version, for now 1 to make it possible to easily do breaking changes to the format
selected list of top level keys form the API, test, settings and environments
environments will patch all the environments in all fields
support for patching a single environment or multiple environments using a regular expression
support for patching a specific plan, or all plans matched by a regular expression and one or more of it’s environments

For matching plans and environments re.fullmatch will be used.

That the unique environment name is not yet implemented in the API. It will be required to be able to patch specific environments.

# Versioning in case we need breaking changes
version: 1

# set "plan_filter" field in "test.tmt"
#
# * the "plan_filter" value will be forced to "enabled: true"
# * other content will be preserved
test:
  tmt:
    plan_filter: "enabled: true"

# patching of "settings" field to enable multi-host pipeline
#
# * the "type" value will be forced to "tmt-multihost"
# * other content will be preserved
settings:
  pipeline:
    type: tmt-multihost

# patching all environments
#
# * context dimension "how" will be forced to "full" in all environments
# * other content will be preserved
environments:
  tmt:
    context:
      how: full

# patching only selected environment for all plans
#
# * only "x86_64-env" environment will be updated
# * variable "ARCH" will be forced to "x86_64"
# * other content will be preserved
environment:
  x86_64-env:
    variables:
      ARCH: x86_64

# patching all environments selected by a regular expression
#
# * only environments matching the regular expression "x86_64-.*" will be updated
# * context dimension "how" will be forced to "full"
# * variable "ARCH" will be forced to "x86_64"
# * other content will be preserved
environment:
  x86_64-.*:
    tmt:
      context:
        how: full
    variables:
      VAR1: VAL1

# with secrets (RSA encryption)
#
# * only "x86_64-env" environment will be updated
# * secret "TOKEN" will be forced to the given encrypted value
# * other content will be preserved
environment:
  x86_64-env:
    secrets:
      TOKEN: |
        joTrPXkIVs9mp9Kh88ly1HAE64Ygu5yRxlrPslb8vG7qNA2isRdvhwO5I5+4WhfjNK43q
        HjCdeIc9LmqZHi5cglYiHHjHZYNhDXatOUt+T7fotyb+VMkXrZj8EiHINgggbJH+/lHBU
        YFhyqjBojyTq1TQUl7FiexTfZS2KFU1st5GgPNcxJJQ2g4lcyXuWNFauC5C4PU08mn1mi

# patching only selected plan, all environments
#
# * only plan "/first-plan" will be updated in all environments
# * context dimension "how" will be forced to "full" in all environments
# * variable "VAR1" will be forced to value "VAL1"
# * other content will be preserved
/first-plan:
  environments:
    tmt:
      context:
        how: full
    variables:
      VAR1: VAL1

# patching only selected plans matched by a regular expression, only a specific environment
#
# * only plans matching will be updated in the environment "aarch64-env"
# * context dimension "how" will be forced to "full" in all environments
# * variable "VAR1" will be forced to value "VAL1"
# * other content will be preserved
/prefix-.*:
  environment:
    aarch64-env:
      tmt:
        context:
          how: full
      variables:
        VAR1: VAL1

# patching only selected plans by a regular expression, environments selected by a regular expression
#
# * only plans matching will be updated in the matching environments
# * context dimension "how" will be forced to "full" in all environments
# * variable "VAR1" will be forced to value "VAL1":
# * other content will be preserved
/prefix-.*:
  environment:
    x86_64-.*:
      tmt:
        context:
          how: full
      variables:
        VAR1: VAL1

Implementation

This section describes the implementation details.

Worker

The worker before the creation of the test schedule entry will:

Validate the configuration file with JSON schema
Patch the schedule entry with the configuration

Secrets

Secrets will be encrypted using 4096-bit RSA public-key cryptosystem. Testing Farm will generate on request a unique RSA key pair for the given user and repository. The private keys will never leave the Testing Farm server. The public keys will be easily accessible to each authenticated user using our API. Users will be able to request public keys for any combination of repository and user.

For decryption the specific private key for a repository and the requesting user will be used. The repository uniqueness will provide protection against stealing secrets by forking repositories. The user uniqueness will provide protection against stealing secrets by rerunning a specific test repository.

CLI

The Testing Farm CLI will provide a new command for the users. The command will access the public key for a specific repository and user and use it to encrypt a secret value. The value will be copy-pasted to the configuration by the user. The git URL will be detected from the current git repository, but user will be able to override it. By default the user associated with the current token will be used, but user will be able to override it.

$ testing-farm secret
using repository 'https://gitlab.com/testing-farm/tests' and user 'batman'

enter secret value: (no echo)


encrypted secret for copy-paste:

jotrpxkivs9mp9kh88ly1hae64ygu5yrxlrpslb8vg7qna2isrdvhwo5i5+4whfjnk43q
hjcdeic9lmqzhi5cglyihhjhzynhdxatout+t7fotyb+vmkxrzj8eihingggbjh+/lhbu
yfhyqjbojytq1tqul7fiextfzs2kfu1st5ggpncxjjq2g4lcyxuwnfauc5c4pu08mn1mi

$ testing-farm secret --user packit
using repository 'https://gitlab.com/testing-farm/tests' and user 'packit'

enter secret value: (no echo)

encrypted secret for copy-paste:

xotrpxkivs1mp9kh88ly1hae64ygu5yrxlr+slb8vg7qna2isrdvhwo5i5+4whfjnk437
hjcdeicdlmqzhi5cglyihhjhzynhdxatout+t7fotyb+vmkxrzj8eihingggbjh+/lhbs
yfhyqjbojytq1tqul7fiextfzs2kfu1st5ggpncxjjq2g4lcyxuwnfauc5c4pu08mn1ma

Pre-commit hook

Testing Farm will provide a pre-commit hook which will validate the yaml format via JSON scheme.

Other Features

Testing Farm API will be extended with a boolean field so users can indicate if secrets should be used. By default requests with secrets will be used. This is useful for users like Packit to avoid parsing the configuration file to understand if they should disallow access to the execution. Instead of disallowing the access, when a non-developer opens a pull request, they will set this field to false and Testing Farm will refuse to run the request if secrets would be needed.