Cloud Based (AWS) Elastic Jmeter Load Testing Application (SWARM)

In this age of internet its imperative for any web based application to benchmark itself for high concurrency. As AWS Advanced Technology partners our work includes helping enterprises/start ups embrace AWS for their production as well as testing workloads. Few questions that people have are

1) Is AWS  scalable ? 

2) How many requests/min can an EC2 instance serve ? 

3) What instance class should I choose for my my application ?

4) How many instances should I chose for my application ?

5) Does Auto Scaling actually work ? 

Turns out there are no simple answers to these question as these are very subjective in nature and vary from one application to the other. The only way to test this is by doing a load test.

Jmeter is almost an industry standard for load testing. We can run a test with desired concurrency and duration and write our own test cases through the GUI provided with it. It can provide a summary in the form a RAW log file (JTL) or a table or a graph.

All this is good when you want to run a load test from one machine, but what if you want to run load test from multiple machines ? How would you aggregate the data across multiple machines ?

You must be wondering why would we need to run load test from multiple machines and why not from one machine only ? 

Some things that I have learnt from my experience are :

1) The test should always be run in a distributed nature. When running concurrent connections from a single machine one could easily reach the network/IO limit of a single machine which would add to response time which would not be correct.

2) Since Jmeter creates multiple concurrent threads, the more the threads more would be the CPU contention which would add to the response time incorrectly.

3) You cannot target requests/unit time for your load test as its a function of number of concurrent threads and the server response time.

4) You can simulate only concurrency with jmeter. For example if you select 100 threads then Jmeter would make sure that there are 100 concurrent requests at any given time. Also Jmeter reuses these threads for maximum performance.

5) When doing load testing for an application behind ELB make sure that either ELB is pre warmed (details here) or you use ramp up. Please note that this is required only when the concurrency you are testing for is very high (there are no numbers shared by AWS). To know whether you are reaching the limits of ELB look for ELB 5XX value in the cloudwatch for your ELB.

6) To know which part of your stack is the bottleneck, use a profiler. My favorite is New Relic. It has plugins for almost all softwares.

To try out our product please visit https://swarm.minjar.com/ .