Performance Test Result Analysis — Part 1

In this article, we will talk about how to interpret a performance test. I will divide my article into 2 parts so that you don’t get bored while reading it. =)

In this part, we will learn with which parameters the tests are generally interpreted.

Introduction of Test Environment

The characteristics of the environment where load tests were conducted should be outlined. This includes details about the hardware, software, network configuration, and other relevant factors. Additionally, information about test scenarios and load profiles should be provided.

Determination of Metrics

Metrics to evaluate performance need to be identified. These metrics may include response times, transaction rates, CPU and memory usage, among other performance indicators. Metrics related to user experience, such as availability and accessibility, should also be considered.

Data Collection and Analysis

Data collected during load tests should be gathered and analyzed. This involves continuous monitoring and recording of the identified metrics. Visualizing data with graphs and analyzing trends can help identify performance issues.

Identification of Performance Issues

The analysis process should be used to identify performance issues within the system. These issues may manifest as longer-than-expected response times, high memory consumption, or excessive CPU usage. A thorough investigation is necessary to determine the root causes of these problems.

Reporting of Findings

The results of the analysis should be presented in a detailed report. This report should include identified performance issues, their root causes, and proposed solutions. Additionally, recommendations for future tests may be provided.

We have learned theoretical information until this stage of the article. Now it’s time to learn information documented by the ISTQB organization. =)

According to Performance Test Syllabus, the following data is analyzed in the performance test report.

Status of simulated (e.g., virtual) users

This needs to be examined first. It is normally expected that all simulated users have been able to accomplish the tasks specified in the operational profile. Any interruption to this activity would mimic what an actual user may experience. This makes it very important to first see that all user activity is completed since any errors encountered may influence the other performance data.

Transaction response time

This can be measured in multiple ways, including minimum, maximum, average, and a percentile. The minimum and maximum readings show the extremes of the system performance. The average performance is not necessarily indicative of anything other than the mathematical average and can often be skewed by outliers. The 90th percentile is often used as a goal since it represents the majority of users attaining a specific performance threshold. It is not recommended to require 100% compliance with the performance objectives as the resources required may be too large and the net effect to the users will often be minor.

Transaction failures

This data is used when analyzing transactions per second. Failures indicate the expected event or process did not complete, or did not execute. Any failures encountered are a cause for concern and the root cause must be investigated. Failed transactions may also result in invalid transactions per second data since a failed transaction will take far less time than a completed one.

Hits (or requests) per second

This provides a sense of the number of hits to a server by the simulated users during each second of the test.

HTTP responses

These are measured per second and include possible response codes such as: 200, 302, 304, 404, the latter indicating that a page is not found.

Thank you for reading this article, see you in the next article. =)

To stay updated with my latest posts, follow my blog and visit my linkedin account.

Happy Testing!