Analyze automated test run results

When an automated test run fails, you can view:

  • Details about the root cause of the failure.

  • Details about the build in which a specific run failed.

  • Previous run history, to see when, how frequently, and why the test is failing, and to check whether it is always due to the same problem or different causes.

To find test run details:

  1. In the Backlog, Team Backlog, or Quality module, select the Tests tab.

    View basic details about the various types of tests and their results. Using the tags pane on the right, filter the tests to see the ones that interest you.

    Example: Select the Latest pipeline run tag to exclude automated tests that were skipped or removed from the pipeline's latest run.

    The display includes only manual tests and automated tests that ran as part of a pipeline's latest run.

  2. Click the ID of a specific automated test.

  3. Select the Runs tab.

    Tip: In the test run grid, add the Problem column to see whether the test has a history of unsuccessful runs. For details, see Problematic tests.

  4. Click the ID of a specific failed test run.

  5. In the Details tab:

    1. Hover over the Build status icon to view build failure and details about the build in which this specific run failed, including when the build failure occurred, and the number of tests that passed, failed, and need attention. If at least 1 test fails, then the entire Build is considered as failed.

    2. Click the External report button to navigate to the report in the external system that produced this run, when applicable.

    3. Click the Pipeline links to navigate to the pipeline and to a particular run of the pipeline (marked by #).

      Tip: To add the Pipeline links column to the grid, use the Choose columns button.

  6. In the Error Info section, view the Error message, Error type, and Error details. The Error Info fields are only populated for failed test runs. If the error details are truncated in the fields, click Show all details to display the error details in full view in a new tab.

    When a test fails, the failure cause (stack trace) is sent to the server. ALM Octane users can immediately see the reason.

  7. On the top-right of the Details tab, click Show in <source> to view the test results in the CI server.

  8. In the Previous Runs tab:

    1. View a list of previous test runs and related build failure and error information, including message, exception, and stack trace.

    2. Filter the run history per specified failure or error criteria. For example, you can filter by run status to view only failed runs so you can immediately identify and resolve them. You can also filter by time duration, build status, and substrings in error messages, exceptions, and stack traces.

    3. In the tooltip of the previous run, click the Open external report link to navigate to the report in the external system that produced this run, when applicable.

    4. Click the Pipeline links to navigate to the pipeline run (marked by #) and to the pipeline.

See also: