benchmarking

Testing how long it takes Chromium to open, load a web page, and quit on Debian

Something I've long been meaning to benchmark, but never really got around to, is benchmarking the amount of time it takes on a Raspberry Pi to open a browser, load a page, and quit.

This is a relatively decent thing to benchmark, compared to other raw performance metrics, because it's something that probably 99% of Raspberry Pi users who use it with a GUI will do, with some frequency (well, probably loading more than one page before quitting, but still...).

So I asked on Twitter:

Raspberry Pi Cluster Episode 5 - Benchmarking the Turing Pi

At this point, I've showed you how you can use the Turing Pi as a Kubernetes cluster to run different things. I barely scratched the surface of what's possible with Kubernetes, but I'm planning on doing another series exploring Kubernetes itself later this year. Subscribe to my YouTube channel if you want to see it!

In this post, I'm going to talk about the Turing Pi's performance. I'll compare it to a more traditional Raspberry Pi cluster, my Pi Dramble, and talk about important considerations for your cluster, like what kind of storage you should use, or whether you should run a 32-bit or 64-bit Pi operating system.

As with all the other work I've done on this cluster, I've been documenting it all in my open source Turing Pi Cluster project on GitHub.

Video version of this post

This blog post has a companion video embedded below:

Benchmarking PHP 7 vs HHVM - Drupal and Wordpress

[Multiple updates: I've added results for concurrencies of 1 and 10, results on bare metal vs. VMware instances, tested Drupal 8 vs Drupal 7 vs Wordpress 4.4, and I've also retested every single benchmark at least twice! Please make sure you're read through the entire post prior to contesting these benchmark results!]

tl;dr: Always test your own application, and trust, but verify every benchmark you see. PHP 7 is actually faster than HHVM in many cases, neck-in-neck in others, and slightly slower in others. Both PHP 7 and HHVM blow PHP ≤ 5.6 out of the water.

Skip to benchmark results:

Vagrant web development - is VMware better than VirtualBox?

[Update 2015-08-25: I reran some of the tests using two different settings in VirtualBox. First, I explicitly set KVM as the paravirtualization mode (it was saved as 'Legacy' by default, due to a bug in VirtualBox 5.0.0), which showed impressive performance improvements, making VirtualBox perform 1.5-2x faster, and bringing some benchmarks to a dead heat with VMware Fusion. I also set the virtual network card to use 'virtio' instead of emulating an Intel PRO/1000 MT card, but this made little difference in raw network throughput or any other benchmarks.]

My Mac spends the majority of the day running at between one and a dozen VMs. I do all my development (besides iOS or Mac dev) running code inside VMs, and for many years I used VirtualBox, a free virtualization tool, along with Vagrant and Ansible, to build and manage all these VMs.

Doing Some Benchmarks - Mac Processor Speed

I currently own or use a variety of Macs, and am approaching the end of a 'cycle' of Mac usage, where I need to decided what Mac I'd like to purchase next. Currently, I'm using a 27" iMac at work, an 11" MacBook Air (from work) for travel, and a 24" iMac at home. They're all great computers in their own right, and using Dropbox, MobileMe, and a couple other helper services, I can operate simultaneously on all three Macs, without any hiccups.

So, I'm thinking about getting a new Mac for hardcore development work (web and app), some graphic design, and possible portability. I have an iPad for lighter computing (reading, browsing, email, videos...), so even though the MacBook Air is probably the best thing to happen to a laptop in a very long time, I'm shying away from it as my primary personal computer.

Running Apache Benchmarks: Drupal/Joomla core vs. Static Page Cache

I just discovered (after asking about it in the #drupal IRC channel) the wonderful little program ab, included in an Apache installation. This little nugget does one thing, and does it well: It beats the heck out of your server, then tells you how your server did in terms of page serving. I tested a few different configurations on a dedicated, 4-core, 4 GB RAM server from SoftLayer, and used the following two commands:

1. Download the specified URL 1,000 times, with KeepAlive turned off (each request gets a new http connection):

ab -n 1000 -c 5 http://ip.address.of.site/path-to-page.php

2. Downlaod the specified URL 1,000 times, with KeepAlive turned on (thus allowing the connection to be maintained for as many http downloads as you have set in your httpd.conf file):

ab -n 1000 -kc 5 http://ip.address.of.site/path-to-page.php

I ran these tests a few different ways, and here are the results of the tests I ran with KeepAlive on, with the number of pages per second ab reported listed after the method:

  • Drupal - normal page caching turned on, css/js aggregation, 55kb page – 12.5 pages/sec
  • Joomla - no page caching (disabled due to buggy 1.x caching), 65kb page – 8.2 pages/sec
  • Drupal - boost module enabled, serving up the boost-cached file – 3,250 pages/sec
  • Joomla - custom page caching system enabled, serving static html file – 2,600 pages/sec

Speed boost due to caching: ~250x faster!