Testing Farm release 2024-02.1

Testing Farm release 2024-02.1 is deployed 🎉.

💔 Breaking Changes

The multihost testing from now uses tmt run provision --update-missing, see docs. This means it will update the image attribute in provision step with the environments[].compose API field only if it is not set in the plan. This might break your current multihost plans. Please update your plans accordingly.

🚀 Upgrades

Testing Farm CLI tool updated from v0.0.16 to v0.0.17, please update your installation.

🎯 Highlights

Created RFD2 for the Testing Farm in config repo which allows configuring secrets in the repository(TFT-1813). Feel free to provide feedback to this planned major feature of Testing Farm.

Added support by selecting tmt tests by name via the --test option in CLI or test.tmt.test_name field in the API (TFT-2421).

RHEL-9.5.0 available on Red Hat ranch (TFT-2434).

CentOS Stream 10 available on Public ranch (TFT-2442).

Multi-host now uses tmt run provision --update-missing improving the multi-host testing by filling in only missing compose (TFT-2379).

Testing Farm now prints a warning when how: artemis is used in provision step (TFT-2306) and crashes when anything else other than how: container or how is unspecified (TFT-2320).

Fedora CI docs now document Testing Farm as their testing infrastructure (TFT-938).

Added support for configuring how many plans should be executed in parallel via the --parallel-limit option in the CLI or settings.pipeline.parallel-limit field in the API (TFT-2262).

Automated importing of updates-nightly composes (TFT-2401).

Added an artifacts link to results viewer page of a running TF test (TFT-2405).

Xunit is generated also in case of pipeline errors (TFT-544).

Time attribute is now show in xunit and results viewer (TFT-2007).

🐞 Bugfixes

Test environment variable TESTING_FARM_REQUEST_USERNAME is now available again (TFT-2433).

Leakage of secrets in testout.log was fixed (TFT-2462).

Fixed a bug where volumes from deployments in EKS clusters were not tagged (TFT-2313).

Fixed showing incomplete pipeline.log file in the artifacts (TFT-2451).

Fixed incorrect Beaker filter for virtualization.is-supported and aarch64 (TFT-2453).

Fixed an issue with artifact artifaction at the end of the pipeline (TFT-2413).

🫂 Contributors

We would like to thank the following external contributors for contributing to Testing Farm.

  • Inessa Vasilevskaya — in Testing Farm CLI, arch is passed into tmt context (cli!97).

📦️ Packages

List of important packages bundled in the worker image.

❯ podman run --entrypoint rpm quay.io/testing-farm/worker:2024-02.1 -q tmt standard-test-roles ansible-core podman beakerlib | sort | uniq
ansible-core-2.14.11-2.fc38.noarch
beakerlib-1.29.3-2.fc38.noarch
podman-4.9.3-2.fc38.x86_64
standard-test-roles-4.11-2.fc38.noarch
tmt-1.31.0-1.fc38.noarch

🔎 Stats

Testing Farm is on track to surpass 1.1M requests per year \o/.

Error rate in February was approximately 2.88%.

Around 98k testing requests were recorded in February.