Testing Farm release 2023-05.1

Testing Farm release 2023-05.1 is deployed 🎉.

Upgrades

Highlights

  • Testing Farm CLI reserve command available, for easy provisioning of a single machine from Testing Farm. (TFT-2073)

  • Non-RHEL teams are onboarding to Testing Farm, see for example strizmi-kafka-operator.

  • Support for customizing global timeout for a request. Default is 12 hours. (TFT-1763)

  • Support for private GIT repositories, the username and password can be passed directly in the git url and are properly treated as secrets. (TFT-1656)

  • Support for tests filtering via the API request. (TFT-1993)

  • Support for installing only certain packages from a repository. (TFT-2070)

  • Support for merge request refs for GitLab and GitHub. (TFT-1578)

  • Metrics for average queue time available for Red Hat and Public Ranch available. (TFT-1764)

  • Kickstart support for Beaker provisioner, on par with kickstart support in the Beaker XML. (TFT-1395) (thanks to Martin Litwora / sst_upgrades)

  • Support for different location of tmt metadata root in the git repositories. (TFT-1706)

  • Artemis can now detect spot instances pre-maturely terminated. (TFT-1888)

  • TESTING_FARM_REQUEST_ID environment variable exposed to tests. (TFT-2044) (Thanks to Lukas Kotek / KVM QE)

  • Support for tuning Beaker watchdog timeout in Artemis. (TFT-1949) (Thanks to Martin Litwora / sst_upgrades)

  • Testing Farm CLI run command available, to easily running small shell snippets in Testing Farm. (TFT-2019)

Bugfixes

  • Excluding of packages for repository and copr build artifacts properly works now. (TFT-977)

  • Compose and arch are not part of the plan name anymore in the JUnit XML. Oculus Improved to show compose and arch below the plan name. (TFT-1810)

  • Multiple fixes for dnf5 which landed in Fedora Rawhide. Zuul PR testing blocked on https://github.com/rpm-software-management/dnf5/issues/549[dnf5/issues/549].

  • Non-printable characters in results-junit.xml are now properly handled. (TFT-2039)

  • Plans with non-standard characters supported. (TFT-1881)

  • Test machine’s hostname, FQDN and reverse DNS records are now properly resolvable. (TFT-1987)

  • Improved stability of aarch64 in AWS by using non-spot instances. (TFT-2080)

Packages

List of important packages bundled in the worker image.

❯ podman run --entrypoint rpm quay.io/testing-farm/worker:2023-05.1 -q tmt standard-test-roles ansible-core podman beakerlib | sort | uniq
ansible-core-2.14.2-3.el8.x86_64
beakerlib-1.29.3-1.el8.noarch
podman-4.3.1-2.module_el8.8.0+1254+78119b6e.x86_64
standard-test-roles-4.10-6.el8.noarch
tmt-1.24.1-3.el8.noarch

Statistics

  • Error rate ~ 3.69% in May

  • ~ 56k testing requests in May

  • Average queued time ~26 second in Public, ~5 minutes in Red Hat