This article is a walk-through of how to use Siege to load (or performance) test a website. In this example the target is the Magento application.
The question is “how many visitors can the target site handle while keeping page maximum load times below 10 seconds”? There are add-on tools that make this simpler but i want to use this example to show the process.
I have a set of URLs where 80% are HTTP and 20% are HTTPS. The idea being that not all traffic is HTTP and/or HTTPS. Use your analytics to find out the true percentage. You can make up the list of URLs based on what you know (from seeing your analytics) visitors most often click on.
http://www.example.com/ https://www.example.com/accessories/a http://www.example.com/accessories/footwear/b http://www.example.com/shop-by-edit http://www.example.com/shop-the-look/c http://www.example.com/stores/a https://www.example.com/accessories/a http://www.example.com/sale http://www.example.com/clothing/pants http://www.example.com/whats-new http://www.example.com/shop-the-look/a http://www.example.com/accessories/b/a http://www.example.com/a https://www.example.com/accessories/b http://www.example.com/shop-the-look/c http://www.example.com/clothing/d http://www.example.com/shop-the-look http://www.example.com/accessories/e https://www.example.com/accessories/f http://www.example.com/clothing
The test is based on the above 80/20 ratio. Here’s the results of a single run over the URLs. I’ve put the above list of URLs into a file called “./urls.txt.http_https”. Tip: redirect the Siege output with a “> /tmp/output” to limit the output on the screen. You only want to do that once you know the tests are working.
siege -f ./urls.txt.http_https -c1 --reps=once Transactions: 2939 hits Availability: 100.00 % Elapsed time: 315.37 secs Data transferred: 53.13 MB Response time: 0.11 secs Transaction rate: 9.32 trans/sec Throughput: 0.17 MB/sec Concurrency: 0.98 Successful transactions: 2923 Failed transactions: 0 Longest transaction: 9.52 Shortest transaction: 0.04
The above shows that if a single visiter visits each of the URLs, they will have downloaded a total of 53.13MB and it would require 2939 individual requests to the target sever.
The same test again with a concurrency of 2.
siege -f ./urls.txt.http_https -c2 --reps=once Transactions: 5878 hits Availability: 100.00 % Elapsed time: 330.41 secs Data transferred: 106.27 MB Response time: 0.11 secs Transaction rate: 17.79 trans/sec Throughput: 0.32 MB/sec Concurrency: 1.96 Successful transactions: 5846 Failed transactions: 0 Longest transaction: 16.06 Shortest transaction: 0.04
The transactions are double as expected. The following is the same test but the URLs are randomised.
siege -f ./urls.txt.http_https -c2 --reps=once -i Transactions: 6002 hits Availability: 100.00 % Elapsed time: 347.45 secs Data transferred: 105.29 MB Response time: 0.11 secs Transaction rate: 17.27 trans/sec Throughput: 0.30 MB/sec Concurrency: 1.89 Successful transactions: 5970 Failed transactions: 0 Longest transaction: 5.88 Shortest transaction: 0.04
The results are not exactly what we’d expect but similar enough to be fine for testing. Our customers are random.
The following is the same test but with 4 concurrent visitors.
siege -f ./urls.txt.http_https -c4 --reps=once -i Transactions: 11149 hits Availability: 100.00 % Elapsed time: 397.65 secs Data transferred: 202.16 MB Response time: 0.13 secs Transaction rate: 28.04 trans/sec Throughput: 0.51 MB/sec Concurrency: 3.68 Successful transactions: 11081 Failed transactions: 0 Longest transaction: 5.64 Shortest transaction: 0.04
The higher we go in concurrency, the more inaccurate our transaction count becomes. However it’s still valid enough because remember we’re basing this all on visitor behaviour which is not consistent.
At this point we know that we can increase the concurrency to the point where the site maximum load times is no more than 10 seconds. We then need to translate this into real-world visiter experiences.
To translate the results in to “real visitors” we need have good analytics. We need to translate the “Transactions” into “visitors” over a given period of time. For this we need to know how long visitors stay on the site for and how often they click on links to different pages within the Magento site.
Here’s where we to do the maths.
Let’s assume our visitors spend 3 minutes on our Magento site and they click on a new link every 10 seconds. By this we know that each visitor clicks a total of “(3 minutes X 60 seconds = 180 seconds) / 10 seconds = 18” time. Each visitor clicks 18 times over a 3 minute period.
We need to know how many transactions make up a single page load. Let’s divide the total number of transactions for the 20 URLs and divide that number by 20 to get the average. We get “2939 / 20 = 147 transactions”. Each page load equals (give or take) 147 transactions.
1 visitor spending 3 minutes on the Magento site causes “18 clicks X 147 transactions = 2,646 transactions”.
2 visitor spending 3 minutes on the Magento site causes “(18 clicks X 147 transactions) X 2 = 5,292 transactions”.
4 visitor spending 3 minutes on the Magento site causes “(18 clicks X 147 transactions) X 4 = 10,584 transactions”.
10 visitor spending 3 minutes on the Magento site causes “(18 clicks X 147 transactions) X 10 = 26,460 transactions”.
Interesting point: The “4 visitor” calculation above results in “nearly” the same number of transactions as in the last Siege test above which was testing for 4 concurrent users. Our maths looks good.
