Tips for Managing Drupal 8 projects with Composer

It's been over a year since Drupal 8.0.0 was released, and the entire ecosystem has improved vastly between that version's release and the start of the 8.3.0-alpha releases (which just happened a couple weeks ago).

One area that's seen a vast improvement in documentation and best practices—yet still has a ways to go—is Composer-based project management.

Along with a thousand other 'get off the island' initiatives, the Drupal community has started to take dependency management more seriously, by integrating with the wider PHP ecosystem and maintaining a separate Drupal.org packagist for Drupal modules, themes, and other projects.

At a basic level, Drupal ships with a starter composer.json file that you can use if you're building simpler Drupal sites to manage modules and other dependencies. Then there are projects like the Composer template for Drupal projects (which Drupal VM uses by default to build new D8 sites) and Acquia's BLT which integrate much more deeply with Composer-based tools and libraries to allow easier patching, custom pathing, and extra library support.

One thing I've found lacking in my journey towards dependency management nirvana is a list of all the little tips and tricks that make managing a Drupal 8 project entirely via Composer easier. Therefore I'm going to post some of the common (and uncommon) things I do below, and keep this list updated over time as best practices evolve.

Adding a new module

In the days of old, you would either download a module from Drupal.org directly, and drag it into your codebase. Or, if you were command line savvy, you'd fire up Drush and do a drush dl modulename. Then came Drush Makefiles, which allowed you to specify module version constraints and didn't require the entire module codebase to exist inside your codebase (yay for smaller repositories and repeatable deployments and site rebuilds!).

But with Composer, and especially with the way many (if not most) Drupal 8 modules integrate with required libraries (e.g. TODO Solarium/Solr/link to issue in search api solr module queue), it's easier and more correct to use composer require to add a new module.

Drupal.org modules don't quite follow semantic versioning, but the way release versioning works out with the Drupal.org packagist endpoint, you should generally be able to specify a version like "give me any version 8.x-1.0 or later, and I'll be happy".

Therefore, the proper syntax for requiring a module this way (so that when you run composer update drupal/modulename later, it will update to the latest stable 8.x-1.x release) is:

composer require drupal/modulename:^1.0

This says "add modulename to my codebase, and download version 1.0 or whatever is the latest release in the 8.x-1.x release series (including alpha/beta releases, if there hasn't been a stable release yet).

Note on version constraints: Two of the most-commonly-used version constraints I see are ~ (tilde) and ^ (caret). Both are similar in that they tell Composer: 'use this version but update to a newer version in the series', but the tilde is a bit more strict in keeping to the same minor release, while the caret allows for any new version up to the next major release. See this article for more details: Tilde and caret version constraints in Composer. See this Drupal core issue for discussion on why the caret is often preferred in Drupal projects: Prefer carat over tilde in composer.json.

Updating modules

Early on in my Composer adventures, I did the reasonable thing to update my site—I ran composer update, waited a while for everything to be updated, then I committed the updated composer.json and composer.lock files and was on my merry way. Unfortunately, doing this is kind of like cleaning a dirty blue dress shirt by washing it in a bucket of bleach—sure, the stain will be removed, but you'll also affect the rest of your shirt!

If you are meticulous about your dependencies, and lock in certain ones that are finicky at specific versions (e.g. composer require drupal/modulename:1.2) or at a specific git commit hash (composer require drupal/modulename:dev-1.x#dfa710e), then composer update is manageable.

But if you're managing a project with many moving parts using more than a dozen contributed modules... be cautious when considering running composer update without specifying specific modules to update!

Instead, what I recommend is a more cautious approach:

  1. See what modules need updating.
  2. Update those modules specifically using composer update drupal/modulename --with-dependencies.

If you had required the module using a release series like drupal/modulename:^1.0, then Composer will update that module—and only that module—to the latest tagged release in the 8.x-1.x branch. And adding --with-dependencies will ensure that any libraries the module depends on are updated as well (e.g. if you update the Search API Solr module, the Solarium dependency will also be updated).

Another quick tip: In addition to Drupal core's update module functionality and drush pm-updatestatus, you can use Composer's built-in mechanism to quickly scan for outdated dependencies. Just use composer outdated. This will show you if Drupal core, contrib modules, or any other dependencies are outdated.

Removing modules

This one is pretty easy. To remove a module you're no longer using (be sure it's uninstalled first!):

composer remove drupal/modulename

Older versions of Composer required a flag to also remove module dependencies that aren't otherwise required, but modern versions will remove the module and all it's dependencies from your composer.json, composer.lock, and the local filesystem.

Requiring -dev releases at specific commits

From time to time (especially before modules are stable or have a 1.0 final release), it's necessary to grab a module at a specific Git commit. You can do this pretty simply by specifying the dev-[branch]#[commit-hash] version constraint. For example, to get the Honeypot module at it's latest Git commit (as of the time of this writing):

composer require drupal/honeypot:dev-1.x#dfa710e

Be careful doing this, though—if at all possible, try to require a stable version, then if necessary, add a patch or two from the Drupal.org issue queues to get the functionality or fixes you need. Relying on specific dev releases is one way your project's technical debt increases over time, since you can no longer cleanly composer update that module.

Regenerating your .lock file

Raise your hand if you've ever seen the following after resolving merge conflicts from two branches that both added a module or otherwise modified the composer.lock file:

$ composer validate
./composer.json is valid, but with a few warnings
See https://getcomposer.org/doc/04-schema.md for details on the schema
The lock file is not up to date with the latest changes in composer.json, it is recommended that you run `composer update`.

Since I work on a few projects with multiple developers, I run into this on almost a daily basis. Until recently, I would find a module, then run a composer update drupal/modulename. Now, I just found that I can quickly regenerate the lockfile without updating or requiring anything, by running:

composer update nothing

Note that some people on Twitter mentioned there's a composer update --lock command that does a similar thing. The docs say "Only updates the lock file hash to suppress warning about the lock file being out of date." — but I've had success with nothing, so I'm sticking with it for now until someone proves --lock is better.

Development dependencies

There are often components of your project that you need when doing development work, but you don't need on production. For example, Devel, XHProf, and Stage File Proxy are helpful to have on your local environment, but if you don't need them in production, you should exclude them from your codebase entirely (not only for minor performance reasons and keeping your build artifacts smaller—non-installed modules can still be a security risk if they have vulnerabilities).

Composer lets you track 'dev dependencies' (using require-dev instead of require) that are installed by default, but can be excluded when building the final deployable codebase (by passing --no-dev when running composer install or composer update).

One concrete example is the inclusion of the Drupal VM codebase in a Drupal project. This VM configuration is intended only for local development, and shouldn't be deployed to production servers. When adding Drupal VM to a project, you should run:

composer require --dev geerlingguy/drupal-vm:^4.0

This will add geerlingguy/drupal-vm to a require-dev section in your composer.json file, and then you can easily choose to not include that project in the deployed codebase.

Commiting your .lock file

The Composer documentation on the lock file bolds the line:

**Commit your application's composer.lock (along with composer.json) into version control.

For good reason—one of the best features of any package manager is the ability to 'lock in' a set of dependencies at a particular version or commit hash, so every copy of the codebase can be completely identical (assuming people haven't gone around git --force pushing changes to the libraries you use!), even if you don't include any of the code in your project.

Ideally, a project would just include a composer.json file, a composer.lock file, and any custom code (and config files). Everything else would be downloaded and 'filled in' by Composer. The lock file makes this possible.

Patching modules

Acquia's BLT the composer-patches project, which is what it says on the tin: "Simple patches plugin for Composer."

Use is fairly simple: first, composer require cweagans/composer-patches:^1.0, then add a patches section to the extra section of your composer.json file:

    "extra": {
        "patches": {
            "acquia/lightning": {
                "New LightningExtension subcontexts do not autoload": "https://www.drupal.org/files/issues/2836258-3-lightning-extension-autoload.patch"
            },
            "drupal/core": {
                "Exposed date view filter fix": "https://www.drupal.org/files/issues/2369119-145_0.patch"
            }
    }

Once you've added a patch, you might wonder how to get Composer to apply the patch and update composer.lock while still maintaining the same version you currently have (instead of running composer update drupal/module which may or may not update/change versions of the module).

The safest way is to run composer update none (a handy trick yet again!), which allows Composer to delete the module in question (or core), then download the same version anew, and apply the specified patch.

Other Tips and Tricks?

Do you know any other helpful Composer tricks or things to watch out for? Please post them in the comments below!

See related: Composer and Drupal are still strange bedfellows.

Comments

Composer outdated would be the nice addition to check the outdated modules.

Remember that each "composer update" should have "--with-dependencies" added to it.
"composer update drupal/address" -> updates the Address module.
"composer update drupal/address --with-dependencies" -> updates the Address module and the underlying libraries.

Thanks! I've added that in—I mentally note to do that with certain modules which I know have deps, but it's probably a safe enough thing to do by default when updating a module.

Hello, is possible that you create a drupal site with the composer way and then make another one drupal site requiring it as a dependency to the other site?
So Site A has a composer with a lot of modules and you want to require Site A as a dependency in your Site B... is a crazy idea?

Thanks for the great writeup! One thing i still haven't wrapped my head around, is it also be possible to update `drupal/core`? And what would be the necessary procedure? Set the site to the maintenance mode... Afterwards `composer update drupal/core --with-dependencies` ...Will that composer command update the core directory in docroot, the files in the root of docroot as well as the vendor directory inside the parent directory (in case of using DrupalVM docroot setup?)... and afterwards run the update.php? So not only the updates of the modules but also those of core could be entirely covered by Composer?

Great tips! Only thing I think is missing that I've noticed has come up with a lot of new Drupal developers is how to properly remove a module in Drupal 8 managed with Composer.

Is it worth adding that after composer require you still also need drush en modulename -y to enable it and trigger any install hooks? Is there a better way? (This doubling up is one of the things I find irritating about using composer, mainly because it's another step to remember when you actually install to production.)

One challenge I constantly have is when using drupal-composer/drupal-project to install a new site, neither Drush nor Drupal Console work for me out of the box. Install directions for both Drush and Drupal Console are confusing because they are not intended for this exact setup. And nobody mentions this in their Composer for Drupal tutorials. Do you follow any steps to make make these two command-line tools usable when installing Drupal using drupal-composer/drupal-project?

I tried your recommendation, but did not figure it out. Here is what I did for Drush:

A. Add "drush/drush": "~8.0", to your composer.json right after drupal/core. Run: composer update drush/drush. Test: bin/drush.launcher. If you get help commands, drush is working. All you need to do is update your alias.

B. To update the alias: Change to home directory: cd ~ Nano .bashrc Paste something like: alias drush="~/public_html/PROJECT_ROOT/(vendor?)/bin/drush.launcher" into the end of the file and save. Close your terminal window and login again to reboot it. (I'm sure there's a shortcut here.) Navigate to “web” and try drush

I'm sure there are shortcuts to this process, but with my limited command-line knowledge, this is what worked for me.

Thanks, this article will be helpful as I work more in Composer. If I want to require the latest dev version of a module, but not a specific version number, how do I do that? You mention drupal/modulename:^1.0 , is that how? (it doesn't mention "dev" so I wasn't clear)

So this gets closest to make the garbage forced upon us usable but it still doesn't actually get there: composer require drupal/webform:^5.0 says The requested package drupal/webform could not be found in any version, there may be a typo in the package name. All in all, I just want to install webform. It's maddening.

A clear and well-written article! You are a great writer. Thanks for this, and your "Composer and Drupal are still strange bedfellows" companion post.

I've just submitted this comment to https://www.jeffgeerling.com/blog/2017/composer-and-drupal-are-still-strange-bedfellows as well, but it might be good to have here for anyone else for reference if they come here first...

I've been delving into composer's source code (to track down this issue: https://www.drupal.org/node/2893017 ), and checked out the different between composer update none, composer update nothing and composer update --lock:

  1. composer update --lock is explicitly supported & documented: https://getcomposer.org/doc/03-cli.md#update
  2. composer update nothing works because 'nothing' doesn't match any packages, and is even explicitly recognised, to avoid debug messages about that: https://github.com/composer/composer/blob/a3f2b7bb92a54f4e25b2d0998f00936ecac6f933/src/Composer/Installer.php#L1332
  3. composer update none also works because 'none' doesn't match any packages, but will probably still show a debug message in verbose output about there not being any matching packages.

So, basically, use composer update --lock :-)

Also, semantically, 'composer update --none' most closely describes what is actually happening (and what you are trying to do). 'composer update nothing' or 'composer update none' gives the impression you don't want to update anything (which would be redundant) when actually you want to update the lock file.