AI Analysis

Testing Farm can automatically analyze test requests using AI and provide a summary of the root cause for errored requests and test failures.

This feature is currently in prototype and is being fine-tuned by the development team. It is only available on the public ranch and currently analyzes errored requests and failed tests.

Viewing AI Analysis

When AI analysis is available for your request, the Results Viewer will show an AI Summary toggle in the header.

Clicking the toggle displays an AI-generated summary that explains:

  • What went wrong with the request or test

  • The specific error encountered (with log references)

  • Whether the failure matches a known outage tracked in Jira

For failed tests, the analysis covers individual test results and provides insights into why specific tests failed.

The AI summary is generated automatically and may contain inaccuracies. Always verify the analysis against the actual test logs.

Availability

  • Errored requests are analyzed (e.g. provisioning failures, guest setup issues, Ansible playbook errors)

  • Failed tests are analyzed to identify root causes of test failures

  • Only the public ranch is supported

  • Analysis runs continuously and results are typically available within a few minutes after the request completes

  • The AI model used is Claude Sonnet 4.6