Performance Testing 101: What is Performance Testing?

This is the second installment in a multi-part series to address the basics of performance testing applications. It’s for the beginner, but it is also for the experienced engineer to share with project team members when educating them on the basics as well. Project Managers, Server Team members, Business Analysts, and other roles can utilize this information as a guide to understanding performance testing process, the typical considerations, and how to get started when you have never embarked on such a journey.

Performance Testing 101: What is Performance Testing?

• Load Testing – typically the most fundamental type of performance testing.  The idea of load testing is a short-term test of performance under real-world conditions.  The key element used in this test is response time.  Using response times, system architects can understand the performance of their site.
• Stress Testing – Is used to find stress points and maximum capacity of an application.  Additionally this test can demonstrate how the application behaves under high stress.  Does the application crash and become unresponsive or do response times increase to a point where they frustrate users.  Monitoring server utilization is usually the key element in this type of test.
• Duration Testing – Is used to see how the application performs over time.  The time frame can easily be hour, days, or even longer.  Some applications may have memory issues that only appear after some amount of time.  Other issues may be slower response times as the database grows.
• Component Testing – It is also possible to focus on certain components or tiers of the application.  This can be helpful early on in the development process to find out if the application will be able to scale to anticipated volumes.

Definitions

Before more details of the performance testing process are presented a list of definitions should be covered.  Often, key terms have different meanings in performance testing then they might in the business.

Load Test – Emulating the load on a system under peak (but realistic) usage conditions and normal user/system interaction.  Identifies the number of users/transactions that can effectively utilize the system within given thresholds. Thresholds consist of a combination of transaction times and system resource utilization. It is not a stress test of any of the components of a system. It’s very important to establish the difference between a load test and a stress test at the beginning of a project with all stakeholders to get them all on the same page. Many people consider these terms as interchangeable, and they are not. The best way to differentiate them is that a Load Test is the perfect storm that could actually happen, while a stress test is an artificially heavy storm intended to find the first point of failure.

Stress Test – Generating load on a system under maximum conditions to determine the failure point. The goal is to ramp the load until failure and identify bottlenecks and scalability limitations. Without the data from a true Load Test (see definition above), this data about the first point of failure may not be as useful for making a business decision about risk and deployment to production.

Vuser – A Virtual User (script) created using a development tool in LoadRunner. The Vuser/script executes during a scenario replaying actions that mimic the user experience. Vuser scripts can measure and record the performance of application components. Scripts can also be used to produce system-to-system transaction based loads.

Business Process – The execute path a user would take when using various functions of the application, or the “start to finish” code execution of a system process.  A script tries to simulate the business process.

Iteration –The loop that a vuser performs within the script/business process.  This may include the entire script/business process or subsection of it.  This is synonymous with what can be thought of as a transaction or completed business process.

Pacing – The amount of time an iteration should take to execute.

Think Time – The amount of time a user would normally wait between page/server requests. Presented as an average.

Transaction Timings – End-to-end measurement of one or more user/system activities within a business process.  This is reported in the LoadRunner results as end-user response time.

Transaction – Similar to transaction timings.  Encapsulates the page(s)/server request(s) and is the smallest measurable increment in LoadRunner.  The hierarchy of transactions is part of the iteration, the iteration is part of the script/business process, and the script/business process is part of the scenario.  A key aspect of the final report will be the transactions per second which will help determine load levels.  This should not be mistaken for a transaction or transaction code in SAP.

Concurrent Users – Users accessing the application at various places within the business process. These users are concurrently using resources but may not be executing the exact same functions at exactly the same moment in time. This is a concept that can be confusing to many people the first time they attempt to create goals and objects for volume levels and concurrent users. There are actually three types of concurrency: Application level, business process level, and transaction level. Application level concurrency is the number of users that would typically be on the system doing “something” – i.e. causing activity and consuming resources. You may also need to define transaction level concurrency, which is how many transactions (form submissions or writes to the database) occur simultaneously. Application level concurrency is typically the easiest and best option to choose when profiling the user activity of an application.

In the next installment of this blog series we will discuss planning a performance test.