vagrant

Tips for a better Vagrant-based development workflow

I build and destroy a lot of VMs using Vagrant in the course of the day. Between developing Drupal VM, writing Ansible for DevOps, and testing dozens of Ansible Galaxy roles, I probably run vagrant up and vagrant destroy -f at least a dozen times a day.

Building all these VMs would be a pain, and require much more user intervention, if it weren't for a few things I've done on my local workstation to help with the process. I thought I'd share these tips so you can enjoy a much more streamlined Vagrant workflow as well!

Major improvements to Drupal VM - PHP 7, MariaDB, Multi-OS

Drupal VM - Vagrant and Ansible Virtual Machine for Drupal Development

For the past couple years, I've been building Drupal VM to be an extremely-tunable, highly-performant, super-simple development environment. Since MidCamp earlier this year, the project has really taken off, with almost 200 stars on GitHub and a ton of great contributions and ideas for improvement (some implemented, others rejected).

In the time since I wrote Developing for Drupal with Vagrant and VMs, I've focused on meeting all my defined criteria for the perfect local development environment. And now, I'm able to say that I use Drupal VM when developing all my projects—as it is now flexible and fast enough to emulate any production environment I use for various Drupal projects.

Developing for Drupal with Vagrant and VMs

Many blog posts have outlined the benefits of using VMs (Virtual Machines) for local Drupal development instead of either using native PHP and Apache, or a bundled environment like MAMP, XAMPP, or Acquia Dev Desktop. The advantages of using virtualization (usually managed by Vagrant) are numerous, but in certain cases, you can make a good argument for sticking with the traditional solutions.

If you'd like to take the dive and start using virtualized development environments, or if you're already using Vagrant and VirtualBox or some other VM environment (e.g. VMWare Fusion or Parallels Desktop), how do you optimize local development, and which pre-bundled Drupal development VM will be best for you and your team?

Criteria for the Perfect Local Development Environment

These are the criteria I use when judging solutions for local Drupal development (whether virtualized or traditional):

  • Should be simple and easy to set up
  • Should be fast by default
  • Should be flexible:
    • Should work with multiple providers; VirtualBox is free, but VMWare can be much faster!
    • Should allow configuration of the PHP version.
    • Should work with your preferred development workflow (e.g. drush, makefiles, manual database sync, etc.)
    • Should prevent filesystem friction (e.g. permissions issues, slow file access speeds, etc.)
    • Shouldn't have hardcoded defaults
  • Should be complete:
    • Should work without requiring a bunch of extra plugins or 3rd party tools
    • No extra languages or libraries should be required (why install Ruby gems, npm modules, etc. unless you need them for your particular project?)
  • Should be Free and Open Source
  • Should include all the tools you need, but allow you to disable whatever you don't need (e.g. XHProf, Apache Solr, etc.)
  • Should work on Windows, Mac, and Linux with minimal or no adjustment
  • Should be deployable to production (so your local dev environment matches prod exactly)

A lot of these points may have more or less importance to a particular team or individual developer. If you're a die-hard Mac user and don't ever work with any developers on Windows or Linux, you don't need to worry about Windows support. But some of these points apply to everyone, like being fast, simple, and flexible.

NFS, rsync, and shared folder performance in Vagrant VMs

It's been a well-known fact that using native VirtualBox or VMWare shared folders is a terrible idea if you're developing a Drupal site (or some other site that uses thousands of files in hundreds of folders). The most common recommendation is to switch to NFS for shared folders.

NFS shared folders are a decent solution, and using NFS does indeed speed up performance quite a bit (usually on the order of 20-50x for a file-heavy framework like Drupal!). However, it has it's downsides: it requires extra effort to get running on Windows, requires NFS support inside the VM (not all Vagrant base boxes provide support by default), and is not actually all that fast—in comparison to native filesystem performance.

I was developing a relatively large Drupal site lately, with over 200 modules enabled, meaning there were literally thousands of files and hundreds of directories that Drupal would end up scanning/including on every page request. For some reason, even simple pages like admin forms would take 2+ seconds to load, and digging into the situation with XHProf, I found a likely culprit:

Resizing a VirtualBox Disk Image (.vmdk) on a Mac

Every now and then, a project I'm managing through Vagrant (using either a box I built myself using Packer, or one of the many freely available Vagrant Boxes) needs more than the 8-12 GB that's configured for the disk image by default. Often, you can find ways around increasing the disk image size (like proxying file storage, mounting a shared folder, etc.), but sometimes it's just easier to expand the disk image.

Unfortunately, VBoxManage's modifyvm --resize option doesn't work with .vmdk disk images (the default format used with Vagrant boxes in VirtualBox). Luckily, you can easily clone the image to a .vdi image (which can be resized), then either use that image, or convert it back to a .vmdk image. Either way, you can expand your virtual disk image however large you want (up to the available free space on your physical drive, of course!).

Here's how:

1 - Convert and resize the disk image

First, vagrant halt/shutdown your VM, then in Terminal or on the command line:

rsync in Vagrant 1.5 improves file performance and Windows usage

I've been using Vagrant for almost all development projects for the past two years, and for projects where I'm the only developer, Vagrant + VirtualBox has worked great, since I'm on a Mac. I usually use NFS shared folders so I can keep project data (Git/SVN repositories, assets, etc.) on my local computer, and share them to a folder on the VM, and not suffer the performance penalty of using VirtualBox's native shared folders.

However, this solution only scaled well to other Mac and Linux users with whom I shared development responsibilities. Windows users were left in a bit of a lurch. To extend an olive branch, I hackishly added SMB support by installing and configuring an SMB share from within the VM only on windows hosts, so Windows devs could mount the SMB share and work on files in their native editors.

Vagrant - NFS shared folders for Mac/Linux hosts, Samba shares for Windows

[Edit: I'm not using rsync shared folders (a new feature in 1.5+) instead of SMB/NFS - please see this post for more info: rsync in Vagrant 1.5 improves file performance and Windows usage].

[Edit 2: Some people have reported success using the vagrant-winnfsd plugin to use NFS in Windows.]

I've been using Vagrant to provision local development and testing VMs for a couple years, and on my Mac, NFS shared folders (which are supported natively by VirtualBox) work great; they're many, many times faster than native shared folders. To set up an NFS share in your Vagrantfile, just make sure the nfs-utils package is installed on the managed VM, and add the following:

    config.vm.synced_folder "~/Sites/shared", "/shared",
      :nfs => !is_windows,
      id: "shared"

VirtualBox, Vagrant, and Ansible: local development environment prowess

I recently gave a presentation titled Local Development Environments - Vagrant, VirtualBox, and Ansible. The presentation explains the importance and efficacy of using (and how to use) local Virtual Machines under VirtualBox, managed with Vagrant, and provisioned with Ansible, especially in comparison to using more traditional tools like WAMP, MAMP, or other prepackaged server solutions.

Local Development Environments

By the end of the presentation, you'll hopefully see how easy—and powerful—it is to create virtual machines for local web and application development.

Fixing SSH unknown error when provisioning a Vagrant VM with Ansible

While getting a local VM managed by Vagrant to work with Ansible for provisioning, I kept getting errors like the following:

fatal: [solr] => SSH encountered an unknown error during the connection. We recommend you re-run the command using -vvvv, which will enable SSH debugging output to help diagnose the issue

[vm-name-here] : ok=0    changed=0    unreachable=1    failed=0

FATAL: no hosts matched or all hosts have already failed -- aborting

Ansible failed to complete successfully. Any error output should be
visible above. Please fix these errors and try again.

It seems that Ansible is unable to connect to the VirtualBox host via SSH because the entry for 127.0.0.1 in my ~/.ssh/known-hosts file is set for my local computer, and not for any VMs. To work around this limitation, I created a new file, ~/.ssh/config, with the contents:

Host 127.0.0.1
        StrictHostKeyChecking no
        UserKnownHostsFile=/dev/null

Now, when Ansible tries connecting during provisioning, it doesn't check the host key for localhost, and provisioning succeeds.