Running Apache Benchmarks: Drupal/Joomla core vs. Static Page Cache

I just discovered (after asking about it in the #drupal IRC channel) the wonderful little program ab, included in an Apache installation. This little nugget does one thing, and does it well: It beats the heck out of your server, then tells you how your server did in terms of page serving. I tested a few different configurations on a dedicated, 4-core, 4 GB RAM server from SoftLayer, and used the following two commands:

1. Download the specified URL 1,000 times, with KeepAlive turned off (each request gets a new http connection):

ab -n 1000 -c 5 http://ip.address.of.site/path-to-page.php

2. Downlaod the specified URL 1,000 times, with KeepAlive turned on (thus allowing the connection to be maintained for as many http downloads as you have set in your httpd.conf file):

ab -n 1000 -kc 5 http://ip.address.of.site/path-to-page.php

I ran these tests a few different ways, and here are the results of the tests I ran with KeepAlive on, with the number of pages per second ab reported listed after the method:

  • Drupal - normal page caching turned on, css/js aggregation, 55kb page – 12.5 pages/sec
  • Joomla - no page caching (disabled due to buggy 1.x caching), 65kb page – 8.2 pages/sec
  • Drupal - boost module enabled, serving up the boost-cached file – 3,250 pages/sec
  • Joomla - custom page caching system enabled, serving static html file – 2,600 pages/sec

Speed boost due to caching: ~250x faster!

Now, this is not a comparison between Joomla and Drupal speeds, as the two sites are completely different, and have a vastly different purpose/scope. The point is this: caching your dynamic website to static html pages will increase your server capacity significantly. I'll state it differently:

There is nothing more effective to speeding up your website for anonymous visitors than using a static page cache.

No matter what kind of host you end up using, make sure you turn on and enable as much caching as you possibly can; your website could load 250x faster, and your server could survive being on the frontpage of digg, slashdot and yahoo at the same time!


Also read other resources on Midwestern Mac, LLC, on server optimization:

Comments

In terms of Mb per second
(55/1024)*3250 = 174.6
(65/1024)*2600 = 165.0

When levels are that high, the bottle-neck can be the network. Which brings me to my next point Gzip! You can usually expect a file that is 1/4 the size if it's compressed.
((55*.25)/1024)*3250 = 43.6

BTW what are the results with KeepAlive turned off?

> There is nothing more effective to speeding up your website for anonymous
> visitors than using a static page cache.
A reverse HTTP proxy like squid or varnish would boost performance even further, but you might call a reverse proxy a "static page cache" as well, so your statement holds.

@ mikeytown2: Most definitely! I have gzip enabled in the Apache configuration, but I'm not 100% sure it's working correctly; a few files always show up as not-gzipped, according to Yslow... But you are correct—there is a lot of potential for saved bandwidth. Luckily, this server's operating on a 1 Gbps port :-)
The results with KeepAlive off were about 15% slower. KeepAlive definitely helps; probably saves a bit of the CPU thrashing.
@ Frando: I haven't ever considered a reverse proxy, but yes, that would be just as helpful (and could be even more so, depending on how geographically diverse your site users are).


Drupal - normal page caching turned on, css/js aggregation, 55kb page – 12.5 pages/sec

That's a very strange result. Even on pretty bad servers, I get numbers for Drupal with normal page caching that are 10x higher. 12.5 request/s are more in range of my results with no page cache (4.5-7 requests / second) (http://heine.familiedeelstra.com/varnish-versus-drupal-page-cache).

A problem with reverse proxies (and maybe boost?) is that you need to not just use ab to test the server, but also your brain.

For instance, on a recent client site, someone installed varnish with impressive ab results, but anonymous users still got rather sluggish performance, and the load didn't decrease much.

Turns out the guy used a rather naive approach to performance testing: no regard for cookies set by Drupal!

At the minimum, use pressflow to prevent session cookies for anonymous users and make sure to ALWAYS test with the has_js cookie set. (Use the -C switch).

"Drupal - normal page caching turned on, css/js aggregation"

css/js aggregation will have little effect in this test since ab only pulls the HTML document, not any referenced files.

"your website could load 250x faster,"

Oops, I'm sure that you didn't mean to write that. The HTML document usually accounts for only about 15% of page load time so improving that by 250x means only a 15% improvement in page load time.

@ dalin: True, true... I could say, though, that I could server up 250x more page requests... page load time might only be milliseconds faster (although if I had 2,000 visits in one second, the server would probably feel 250x faster for the last 1,500 visitors!).

@ Heine: For a while, I had anonymous cookies turned off, but was experiencing a few weird bugs with that; it's something to consider again. Also, our Urchin traffic monitor requires a cookie to be set, and I was told from above that we must use it (it helps track 'absolute unique visitors,' which is some sort of godly and amazing marketing metric ;-).

Could the slowness be due to the fact that the site is running something like 60 modules? During *that* test, the CPU was hit hard, with an average of 99.9% utilization.

Wow, 60 modules :) Without APC or a similar cache PHP has to slog through an awful lot of files.

Unless most of these modules implement a boot hook, you should see a big difference between normal and aggressive page caching (In my sample, there was almost no difference).

Quite true - doesn't Drupal load every .module file (but not necessarily any of the other module files) on each page load, during the bootstrapping process? That would definitely be the cause of a bottleneck, especially without some major caching going on.

I've never used agressive caching, but am interested in trying it out. I just wish it were compatible with the page view counter/statistics module :-(

With boost, you can at least still have that small database connection...