Performance testing with JMeter

What is a Performance test?

Performance testing in software quality assurance is the general practice of subjecting a system to various test workloads to determine its performance in terms of responsiveness, scalability, stability, reliability, and resource usage. Conducting a proper performance test will eliminate any bottleneck in the system.

In a performance test, we focus on three main aspects of a system

• Stability: How much load the system can handle.
• Speed: How fast the system responds, which is determined by the Responds Time.
• Scalability: System’s stability to diversifying loads.

Why do we need a performance test?

In the software development world, a functional product with a lot of features is not the only milestone. The system should be fast, smooth, and have good endurance to handle multiple concurrent users and provide a more than satisfactional user experience. The intention of this test is not to find any bugs but to identify any performance bottlenecks.

The results of a performance test can be used to provide stakeholders with information like the system’s response time, stability, and scalability to workloads. A performance test would reveal any modification that needs to be done to the system or its deployment before putting it on the market.

Here is a common list of problems that a product would have in terms of performance:

• Poor responsive time: This is the time taken for the system to provide an output to user interaction.
• Long load time: This is the time taken to initially load the system. Should be minimum as possible.
• Poor scalability: Inability to cater to a large load of users at a given instance.
• Performance bottlenecks: These are the factors that degrade the system’s performance. These include poor CPU, memory, network utilization, OS limitations, and disk usage.

Types of Performance tests

• Load testing- Check how the system would perform to an expected user load.
• Stress testing- Test the application under extreme workload to see its behavior to high traffic.
• Endurance testing- This is done to check if the system handles the expected workload for a long period of time.
• Spike testing- Test how the system reacts to a sudden large spike in load.
• Volume testing- In this testing, the database would be populated with a large amount of data, and monitor the system’s behavior.
• Scalability testing- Tests how scalable your system be in case of increased user load.

Introduction to JMeter

Apache JMeter is a 100% pure Java application, an open-source project designed to conduct performance testing on web applications. JMeter can be used to test the performance of static or dynamic content. It can simulate heavy loads to test applications or servers, to analyze performance under varying loads.

JMeter has the capability to test different applications, servers, or protocols:

• Web- HTTP / HTTPS
• FTP
• Mail- SMTP(S), POP3(S), and IMAP(S)
• Message-oriented middleware (MOM) via JMS
• Native commands or shell scripts
• TCP
• Java Objects

JMeter provides a GUI, CLI, or scripts to build and execute tests. The ability to add plugins to JMeter provides unlimited test capabilities.

To learn more about JMeter visit the official page here.

Basic setup

Requirements:

• Java Virtual Machine (JVM).
• A multi-core CPU with 4 cores or more is suggested, due to the multi-threaded nature of Jmeter.
• About 16 GB RAM, to run heavy loads in testing.

The Jmeter binary can be downloaded here.

Unzip the file, navigate into the bin/ folder, and depending on the OS open the Jmeter executable.

• Windows: Double click on the jmeter.bat file.
• Mac/ Linux: Open a terminal in the directory and enter “sh jmeter.sh”

Perform your first load test with JMeter

Enough of technical details, let us get our hands dirty. Let us build a simple load test with JMeter. As we discussed earlier, a load test will test the performance of an application to an expected number of concurrent users.

Step 1

Open the JMeter GUI.

Step 2

Right-Click on “Test Plan” and select the “Add” option. Immediately you will see a list of options. To perform a load test we need to simulate a user load. This is can be achieved by adding “Threads (Users)”. A thread group can be used to specify the number of users that would access the application at once. Further, we can also specify the number of requests a single user would send sequentially.

Step 3

Now let us create a request for an application. Here I am going to send an HTTP GET request to www.google.com. To do this we need to add a sampler to the thread group. Right-Click on Thread Group, Select ADD, Select “Sampler”.Immediately you will see a list of requests that JMeter can simulate. Finally, Select “HTTP request”.

Step 4

Your test is ready to be executed. But how can you see the result? To view a result of a test we need to add a Listener. Right-Click on your thread group, select Add and click on “Listener”. Now you can select any listener type you want to visualize the test results. For now, we will be select the “View Results Tree”.

Step 5

Here we go, we now have a functional load test. Save the test execute the test by clicking on the green play button.

Viewing results:

Running on CLI

As we saw in the above section, running a test on Jmeter is very much straight forward in the GUI, but did you know that JMeter discourages the use of GUI to execute tests. The GUI is only for test creation and debugging. The main reason is that GUI takes much more computational power than a CLI. This could be widely noticed when administering heavy test suites.

To execute a test on CLI, create a test in GUI, save it inside the bin directory, and open a command prompt or terminal.

Windows users;

jmeter -n -t <File name of test>.jmx -l <File name to save test results>.jtl

Linux/ Mac users;

sh jmeter.sh -n -t <File name of test>.jmx -l <File name to save test results>.jtl

-n: Open in non-GUI mode

-t: The test file

-l: File to save

Example on Ubuntu:

The generated result file can be open with a desired JMeter GUI Listener.

Generating a result HTML Dashboard

Tired of uploading the results file to a listener? Not to worry, JMeter has the perfect solution. With JMeter besides the results file, we can also generate a dashboard in HTML with the most possible interpretations of the results.

How? Simply add -e -o <Folder to generate web page> to the end of the JMeter CLI command.

Now the command would like;

• For Windows:

jmeter -n -t <File name of test>.jmx -l <File name to save test results>.jtl -e -o <Folder to generate web page>

• For Linux/ Mac:

sh jmeter.sh -n -t <File name of test>.jmx -l <File name to save test results>.jtl -e -o <Folder to generate web page>

-e: Execute after running the test.

-o: Folder to store the generated dashboard.

Example on Ubuntu:

Navigate to the created directory and open the index.html with your favorite browser.

What if there is a login page?

Now the question arises of how we can do a performance test on an application that requires a pre-login. The test needs to capture and store this cookie generated during the login. JMeter provides a solution by adding “HTTP Cookie Manager” into a thread group. Simply right-click on the Thread Group → Add → Config Elements → HTTP Cookie Manager.

When an HTTP Request or the response contains a cookie, the Cookie Manager will store that cookie and will be used in future requests.

As you have seen in this blog I have explained the basics of performance testing and practical demonstration with a tool named JMeter. Learning the concepts behind every type of performance test will help anyone to use the tool more efficiently. The capability to add plugins to JMeter gives developers and QA engineers more abilities and possibilities to come up with efficient test plans.