Services

Artifacts Storage

Testing Farm stores all test artifacts in its Artifacts Storage service. Currently there are two deployments of the artifacts storage.

Public Ranch

Red Hat Ranch

Directory index is not provided here, append /<REQUEST_ID> to the URL to get the results for a specific request, e.g. http://artifacts.osci.redhat.com/testing-farm/34125b2c-22d6-4ac5-b603-6f7b83046da2/

Cloud Costs

Testing Farm provides an easy way how to access reports for cloud costs per Testing Farm user.

This feature is available only for Red Hat employees using 🎩 Red Hat ranch. For Public ranch, the feature is currently not planned.

The cloud costs are being reported only for AWS.

The cloud costs reporting is available via AWS web console.

Logging into AWS

Logging in is required in order to access the cloud cost dashboard.

All employees working in PnT have read-only access to the AWS Cost Explorer. To log in, use the following link.

In case you already have access to AWS, you need to choose the following role:

Account: arr-cloud-aws-core (727920394381)
Role: 727920394381-cloud-costs

And click Sign in button.

Accessing Testing Farm Costs Dashboard

To access the dashboard, use the following link.

BaseOS CI costs are not included in the report. If you are interested in BaseOS CI report, see the following link.

By default, the dashboard is showing the cloud spend for the last 7 days.

testing farm costs

To change the reporting date range, use the Report Parameters → Time → Date Range on the right side.

To see the cloud costs per team, use the Cost and usage breakdown on the bottom.

Changing the Team Name

In case you are using Packit service, it is advised to explicitly define the team name for the reporting. By default, the costs will be reported against Packit service, which interacts with Testing Farm.

See the Packit configuration example - Tag cloud resources in Testing Farm for details.

Service-level agreement

The purpose of SLA is to define the agreed-upon service levels and performance expectations for Testing Farm uptime and error rate. It is intended to ensure a high-quality experience for our users and maintain a reliable testing service.

  • API Uptime: The API uptime is a critical metric that reflects the availability and reliability of Testing Farm API. The agreed SLO (Service-Level Objective) for the API uptime is currently set at 99%. This means that we strive to guarantee that Testing Farm API will be operational and accessible at least 99% of the time.

  • Error Rate: The error rate represents the percentage of test requests that resulted in an 'error' state, indicating that Testing Farm failed to deliver a reasonable result. The agreed SLO for the error rate is currently set at 5%. This means that Testing Farm team aims to ensure that 95% of all the requests complete successfully and yield the expected outcomes.

Error Budget

The error budget is a concept that allows for a certain tolerance in service disruptions while maintaining the desired service levels. In the context of Testing Farm SLA, the error budget represents the maximum allowable deviation from the agreed SLOs for API uptime and error rate. In case we consume all the error budget, we will drop into Error Budget Mode, and will announce it to our customers. To learn more about exceeding error budget, visit this page.

To ensure transparency and accountability, Testing Farm Team continuously monitors the API uptime and error rate of Testing Farm.

This monitoring will be carried out on a 24/7 basis, and data will be collected and analyzed for a period of 28 days.

The data is visible to all Red Hat employees in our Service Health page.