Performance Testing 101: Determining Peak Load – A Case Study

This is the fourth installment in a multi-part series to address the basics of performance testing applications. It’s for the beginner, but it is also for the experienced engineer to share with project team members when educating them on the basics as well. Project Managers, Server Team members, Business Analysts, and other roles can utilize this information as a guide to understanding performance testing process, the typical considerations, and how to get started when you have never embarked on such a journey.

Let’s use a case study to walk through the thought processes used to determine peak load for an application performance test.

It is common that the load on a productive system is not constant over time, and that business activities cause statistical fluctuations as well as more systematic changes in the load.  A typical example often seen is time sheet recording, which most employees tend to do on Friday afternoon.  The peak load from such a usage pattern can be significantly higher than the average load.

Often a system sized and tested for such peak load is defined as a requirement.  Since performance should not degrade on Friday afternoon during peak load time, it is more important to determine what the peak load will be.  If estimates are too low, one risks insufficient hardware sizing, which leads to performance degradation.  If peak load sizing is too high, too much hardware may be purchased and the cost of ownership and maintenance is increased greatly.

An example: When considering peak loads, you might ask what is the highest load measured in req/sec that can be caused by 1000 users.  Theoretically the answer is 1000 req/sec if all 1000 users happen to click at exactly the same time.  However, since end-users work asynchronously, chances that this situation will occur are very small.

The extreme example above demonstrates the need to look at peak loads at different time intervals.  The maximum average load within a certain time interval is observed over increasing time intervals.  Sample measurements for 2000 logged-in users performing time sheet entries could be:

• maximum average over one week: 2 requests/ sec
• maximum average over one hour: 10 requests / sec
• maximum average over one minute: 30 requests / sec
• maximum average over a second: 100 requests / sec

This statistical phenomenon shows that the shorter the time interval, the higher the peak load.  In the above case, the long-term average load during office hours is only two requests / sec.  In the above example, peaks of about 50 times the long-term average load were observed.

One can argue that requests that cannot be served by the application immediately, due to a lack of resources, are queued and processed when the peak time is over.  A peak load of one second would cause performance degradation for a total of only a few seconds.  Based on this argument, it is up to the customer to find a reasonable compromise between the length of the time interval for which some performance degradation can be tolerated and the maximum peak load for which the hardware should be sized.  In the above case the decision was to size the hardware to 30 requests / sec in order to cover load peaks which might last for one minute and accept short intervals of a few minutes during which performance occasionally might be degraded.  Hardware and hardware operation costs of up to 300% were saved in comparison to second level peak load sizing.  This requirement would also be the goal for your peak load test.  If you have not yet discussed these aspects of peak load, you should do so with the end users to evaluate the requirements for your test strategy.

In the next installment of this blog series we will introduce the concepts around creating Business Process Profiles and Server Configuration Profiles.