Speeding up Composer-based Drupal installation

Drupal VM is one of the most flexible and powerful local development environments for Drupal, but one the main goals of the project is to build a fully-functional Drupal 8 site quickly and easily without doing much setup work. The ideal would be to install Vagrant, clone or download the project, then run vagrant up. A few minutes later, you'd have a Drupal 8 site ready for hacking on!

In the past, you always had to do a couple extra steps in between, configuring a drupal.make.yml file and a config.yml file. Recently, thanks in huge part to Oskar Schöldström's herculean efforts, we achieved that ideal by switching from defaulting to a Drush make-based workflow to a Composer-based workflow (this will come in the 3.1.0 release, very soon!). But it wasn't without trial and tribulation!

Before we switched the default from Drush make to Composer, I wanted to get initial build times down so users didn't have to wait for an excruciatingly long time to download Drupal. At first, using all the defaults, it took twice as long to build a Drupal site from a composer.json file or using drupal-project as it did to build the Drupal site from an equivalent make file. The main reason is that Composer spends a lot more time than Drush in resolving project dependencies (recursively reading all composer.json files and downloading all the required projects into the vendor directory).

I thought I'd share some of the things we learned concerning speeding up Composer installs and updates for Drupal (and other PHP projects) in a blog post, so the tips aren't buried in issues in Drupal VM's issue queue:

Use prestissimo

prestissimo is a Composer plugin that enables parallel installations. All you have to do is composer global require "hirak/prestissimo:^0.3", and all composer install commands will use parallel package downloads, greatly speeding up the initial installation of Drupal.

For Drupal VM, the Drupal download time went from 400 seconds to 166 seconds—more than 2x faster for the Composer installation!

Make sure XDebug is disabled

This one should be rather obvious, but many times, developers leave XDebug enabled on the CLI, and this slows down Composer substantially—sometimes making installs take 2-4x longer! Make sure php_xdebug_cli_enable is 0 in Drupal VM's config.yml if you have xdebug installed in the installed_extras list.

(If using Vagrant) Use vagrant-cachier

Many Vagrant power users already use vagrant-cachier with their VMs to cache apt or yum packages so rebuilds are quicker (you don't have to re-download frequently-installed packages anymore); but to use it with Composer, you need add one extra bit of configuration in your Vagrantfile:

if Vagrant.has_plugin?('vagrant-cachier')
  ... any other cachier configuration ...

  # Cache the composer directory.
  config.cache.enable :generic, :cache_dir => '/home/vagrant/.composer/cache'
end

We can't use vagrant-cachier's built-in Composer bucket, because PHP isn't preinstalled on the base boxes Drupal VM uses. So we use a :generic bucket instead, and manually point it at the Composer cache directory inside the VM.

Things that didn't seem to help

  • Shallow Git clones: Some people suggested using shallow Git clones (e.g. following this Composer PR), but it didn't make a measurable difference.
  • "minimum-stability": "dev": In the past, setting minimum-stability to dev could speed things up a bit while Composer sorts out the dependency tree (see this post). It seems to not have any measurable impact in this case, though.
  • There are still some other areas ripe for improvement, too—for example, the drupal-packagist project may be able to improve it's caching infrastructure to greatly speed up download times.

Please let me know if there are other tips and tricks you may have that can help speed up Composer—we've almost hit the same build times with Composer that we hit with Drush make files, but make files are still slightly faster.

Comments

A few additional tips:

  • Prefer distributions rather than cloning the source of your dependencies to improve speed, --prefer-dist.
  • Split prod dependencies from dev dependencies and use --no-dev for prod builds.

I'd also add a disclaimer that prestissimo can cause issues with GitHub rate limiting if you're not careful.

GitHub dropped the rate limit (for archive links only) completely.
So it is safe to use `--prefer-dist`.

See https://github.com/composer/composer/issues/4884#issuecomment-195229989

(There is nothing about this in the docs: https://developer.github.com/v3/search/#rate-limit )

prestissimo, awesome, you just saved me hours! Thanks Jeff.

I'd like to point out that the official composer endpoints are now live, and have been in alpha for about a month. We rebuilt everything on Friday with a lot of the changes and feedback we received, and Im almost ready to call it 'beta'.

Please, the more people we have using it, the faster any unusual bugs we are going to shake out.

More information here: https://www.drupal.org/node/2718229