~/blog/jmeter-performance-testing-part-2.mdx
blog9 min#performance#testing

jmeter performance testing: part 2

Apr 8, 2025

I previously shared the beginnings of my journey into JMeter in Part 1:

jmeter performance testing: part 1

If you missed that, the TLDR is that JMeter has a nuanced interface that takes practice. We explored JMeter basics and discussed the setup options used for our specific API service that acts as a mapping translation.

Now, we move on to Part 2! ✌️This time, we’ll look at my usage and experience creating a script to parameterize and automate for the future.

Here, we’ll cover:

  1. 👩🏽‍💻 Usage and Testing
  2. 🤖 Script and Automate

👩🏽‍💻 usage and testing

To get going, I checked how JMeter worked in the GUI with a local server instance and also checked connections to our development service. It was important to me to check that everything worked properly before getting too far.

Then, we defined a plan for our performance testing. We had some example average data from a similar service. In Part 1, we used 50,000 daily requests as an average for simplified math. We’ll continue to use this for consistency. 👍

Before officially running the tests, consider if others will use your service simultaneously. In our case, other team members were implementing and pinging the service.

I informed others using the service with ample notice of the intended testing start and stop times. With a globally distributed team, the overlap was small. That said, it’s important to inform them so they don’t encounter unexpected issues and to avoid requests our tests aren’t capturing.

Next, we’ll cover four key usage areas:

  1. Start with a Plan
  2. Configure the Tests
  3. Test Away
  4. Don’t Forget to Document

Let’s go!

start with a plan

We opted to perform load testing, stress testing, and spike testing. To learn more about these in detail, check this Geeks for Geeks post. In the meantime, here are my generalized definitions:

Load testing checks the average load we expect to receive to ensure we can manage existing requests.

Stress testing pushes those average loads even further to see if we can find a maximum load.

Spike testing adds a lot of stress quickly to see if the server can handle it.

Standard Load testing was recommended by one of our QA team members.

configure the tests

Next, I used the setup options from Part 1. Although it was a bit cumbersome, I created a file for each testing scenario. I did this for two reasons.

  1. I could work in a second JMeter instance to prepare the next test file while the current test runs without interruption. Using the same file and then saving a new version with the latest configurations saved a lot of time, ultimately.
  2. I didn’t want to get my wires crossed with different configurations for each test scenario. With these pre-configured, I could revisit them when needed… Which came in handy when my internet became unstable during development testing.

“So, what were the configurations?” you ask. Let’s cover some rough variations!

🔁 option 1: http request

For each test, this remained generally the same. The only difference was the local http:// request as opposed to the deployment service https:// request.

📋 option 2: http header manager

Another easy one! This remained constant between the sets of local and development service tests, but we used separate values for each environment.

🧐 option 4: extended csv data set config

Yes, I went out of order; bear with me! This was another consistent option. This setup used the same CSV file with thousands of inputs.

⏱️ option 3: constant throughput timer

Here’s where most of the complications came in. Each test had a different throughput amount and/or time range we wanted to test. Below are the details. Remember that the throughput field needs a number for the “samples per minute.”

Again we assume we had an average of 50,000 daily requests with the highest average of 100,000 daily requests for easier reference. I’ve rounded all results for simplification, but you can extend to whatever decimal point range you like.

load testing

stress testing

spike testing

We set up a short “warm-up” period for the server to better simulate a real-world spike among regular traffic in each spike test. Each of my warm-ups lasted 5 minutes and used the average load throughput of 34.7 requests/minute.

It’s worth noting that adding a warm-up period requires an additional thread to set up different throughput and test details.

Spike test warm-up thread configuration in JMeter

For consistency, each spike test was set to last for 10 minutes after the 5-minute warm-up. Below are three options I chose, though you may make different choices based on your expected traffic load or how hard you want to test the system.

standard load testing

After performing all of these, I checked with our QA testing expert to see if I had missed anything. She suggested we perform this test, too! I repeated the average load and highest average load tests with a much longer timeframe.

This is quite a long time to test! In our case, we had others using the API to implement it in the platform to ultimately use the service. For this reason, I tested Friday through Sunday. I checked in a little over the weekend, which is easy to do when you work remotely. 👍

test away

Once everything is set up, run the tests!

I kept an eye on them and did low-key tasks while waiting to check the progress in case of any issues. For instance, my internet cut out during one of my tests! I quickly reset and started the test again.

don’t forget to document

Immediately after finishing the tests, I documented results, stored the test files, and gathered details so anyone could access and run the tests later if desired. Don’t wait!

I added this to a Confluence document with additional information, like which plugin(s) would be needed.

Documentation like this, even if it’s not exciting, will be helpful for your team and your leaders to know what you tested so they can provide feedback and understand what was tested if more testing is needed later.

🤖 script and automate

Despite all that lovely documentation, my teammate and I felt storing test files in the repo would be best for our future selves and teammates.

What would be even better? If our future selves and teammates could run those tests with a simple script, all the better! Let’s talk about implementing automated testing capabilities using parameters.

This is where the real fun began!

user defined variables

The first thing to learn was how to use the CLI and incorporate User Defined Variables. This Stack Overflow response helped get me started.

Again, I toyed with different options and worked through some examples to get comfortable with User Defined Variables. It’s neat to see just how much you can parameterize! Spoiler alert: almost everything!

how to parameterize?

First, we need to add a config element for User Defined Variables.

Adding a User Defined Variables config element

Then, we can add in our key-value pairs. Here’s an example using a key of example-key and a value of example-value.

User Defined Variables with an example key and value

Then, these variables can be plugged into various fields throughout the test! Let’s look at a simple example. We will use the following snippet to introduce our User Defined Variables as a parameterized value:

${__P(example-variable,default)}

The left side within the parentheses is our User Defined Variable name (x-api-key below), and the right is our default value. In the case below, we use foobar. Meaning, if we provide no value, foobar will be used as the value for this header.

A User Defined Variable used in a header with a default value

One important thing I learned during this process is that it is possible to use the full URL in the path instead of setting the protocol, the server name/IP address, and identifying the “Advanced Implementation” setting described here in Part 1.

Using the full URL in the path field of an HTTP Request

consider the options

Once I had thoroughly tested the available parameterization options, I had to consider our goals. I first went overboard and made a single test file that we could run any test we wanted because everything was parameterized, with most of the test numbers defaulting to zero as a safe placeholder.

We discussed this and decided on the best options to parameterize, limiting our files to 3 in total instead of one for every single test scenario. Here’s a sample of the script:

jmeter -n -t <path_to_file.jmx> \
  -Jurl=<url> \
  -Jx-api-key=<optional-api-key> \
  -Jcsv=<path_to_input_file.csv> \
  -f -l <path_to_results_file.jtl>

What’s happening here?

-n - This argument tells JMeter to run in CLI mode

-t - This indicates that the next argument provides a path for the .jmx test file

-Jkey=value - The url, x-api-key, and csv keys were defined within the test plan to accept user input

-f - This argument tells JMeter to force delete existing results files and web report folder if present before starting the test as a cleanup step

-l - This indicates that the next argument provides a path to store the test results in either a .jtl or .csv file

Once a test is completed, you can review the results! Using .csv is an option, but it provides minimal detail. The most informative option uses JMeter GUI mode to review the .jtl results file using the View Results Tree, Summary Report, or Aggregate Report options within the test plan.

create and iterate

I started by creating the three files.

load test

I created a load.jmx file, which contained the default settings for a 60-minute test of the average load.

stress test

The stress.jmx file contained the default settings for a 60-minute test of 3x the average load.

spike test

The spike.jmx file contained the default settings for a 10-minute stress test (50k spike target by default). As with spike testing, there is a 5-minute warm-up before the spike start for a 15-minute total test.

Then, I tested the files thoroughly with all input options I had parameterized to confirm they worked as expected!

Finally, I added these files to the repo and added information on a sample script and how to use it, similar to what I shared above.

I hope that this has been helpful and/or interesting! This was where the fun presented itself in the form of a challenge; it was a joy trying to figure out the parameterization!

📚 further reading

thanks so much for reading! ✨
🧡support the work

All of this (the podcast, the posts, the projects) happens on nights and weekends. If something here saved you a headache, the tip jar is always open.

ko-fi.com/mindiweik →
// related posts
Mar 4, 2025jmeter performance testing: part 1Mar 26, 2026level up: sources, performance & your playground

// comments

Loading comments…

    leave a comment

    // subscribe

    New posts and episodes by email. No spam, unsubscribe anytime.