Drupal VM on Windows - a fast container for BLT project development

AKA "Supercharged Windows-based Drupal development"

tl;dr: Use either PhpStorm or a Samba share in the VM mounted on the host instead of using a (slow) Vagrant synced folder, and use Drupal VM 4.4's new drupal_deploy features. See the video embedded below for all the details!

I've often mentioned that Windows users who want to build modern Drupal sites and apps are going to have a bit of a difficult time, and even wrote a long post about why this is the case (Developing with VirtualBox and Vagrant on Windows).

But for a long time, I haven't had much incentive to actually get my hands dirty with a Windows environment. Sure, I would make sure Drupal VM minimally ran inside Windows 10... but there were a lot of little inefficiencies in the recommended setup process that would lead to developers getting frustrated with the sluggish speed of the VM!

Since a client I'm currently helping is stuck on Windows 7 for the short-term, and can't do internal development on a Mac or Linux (other OSes are not allowed on workstations), I decided to try to set up a project how I would set it up if I had to work in that situation.

Basically, how do you get Drupal VM to perform as well (or better!) on a Windows 7 machine as it does on a Mac, while still allowing a native IDE (like PHPStorm or Notepad++) to be used?

After a couple days of tinkering, I've found the most stable solution, and that is to build Drupal VM 'outside in'—meaning you build your project inside Drupal VM, then share a folder from Drupal VM to Windows, currently using Samba (but in the future, SSHFS might be a viable option for Windows too!).

Here's how I used Drupal VM to run and develop an Acquia BLT-based project locally on Windows 7:

Contents

Install required software

To make this work, you will need to install the following software:

  1. VirtualBox
  2. Vagrant
  3. PuTTY (used for Pageant)
  4. Powershell 4.0+ (if on Windows 7)
  5. Cmder ('full' download)

Create an SSH public/private key pair

Generate an SSH key pair in Cmder

Before doing any work with a project with code hosted on GitHub, Acquia Cloud, Redmine, or elsewhere, you need to create an SSH key pair so you can authenticate yourself to the main Git repository.

  1. Open Cmder (right click and 'Run as Administrator')
  2. Create an SSH key in the default location: ssh-keygen -t rsa -b 4096 -C "[email protected]"
    • Press 'enter' three times to accept the defaults.
  3. Print the contents of the public key: cat C:\Users\[youraccount]\.ssh\id_rsa.pub
  4. Copy the entire contents (from ssh-rsa to the end of your email address).
  5. Add the copied key into GitHub under Profile icon > Settings > SSH and GPG keys.
    • Also add the key into any other relevant service (e.g. Acquia Cloud under Profile > Credentials).

Set your SSH key to run inside Pageant

Puttygen - Import and convert SSH key to ppk

Due to a bug in the current version of Vagrant, you need to use Pageant to allow Vagrant to use your SSH key inside the VM when running commands like vagrant up and vagrant provision.

Also, Pageant requires a specially-formatted version of your key, so you'll need to use puttygen.exe to convert the key before loading it into Pageant.

  1. Install PuTTY.
  2. Open puttygen.exe (inside C:\Program Files\PuTTY).
  3. Select "Conversions > Import key".
  4. In the 'Actions' section, click "Save private key".
  5. In the save dialog, save the file as id_rsa.ppk with type "PuTTY Private Key Files (*.ppk)".
  6. Close puttygen.exe.
  7. Open pageant.exe (this opens a taskbar item).
  8. Right click on the Pageant item in the taskbar, and choose 'Add key', then navigate to where you saved id_rsa.ppk.
  9. You should also ensure Pageant runs on system startup to ensure keys are always available when you need them. See this guide for further instructions on how to have pageant.exe start on boot.

Build Drupal VM

Drupal VM Dashboard page on Windows 7

At this point, we're ready to build a local development environment with Drupal VM.

Note: In the rest of this guide, substitute projectname for your project's own machine name.

  1. Open Cmder (right click and 'Run as Administrator')
  2. Run start-ssh-agent (so your SSH key is loaded and can be used in the VM).
    • When using Pageant this is not strictly required, but once this bug is resolved, you won't need Pageant at all, and you can just use start-ssh-agent.
  3. Clone a copy of Drupal VM to your computer: git clone https://github.com/geerlingguy/drupal-vm.git
  4. Change directories into Drupal VM: cd drupal-vm
  5. Create a shell script to create your project's directory and set it's permissions correctly:

    1. touch examples/scripts/setup.sh
    2. Open your editor of choice (e.g. Notepad++) and edit setup.sh (be sure to use Unix line endings!):

      #!/bin/bash
      mkdir -p /var/www/projectname
      chown vagrant:vagrant /var/www/projectname
      
  6. Create a config.yml file: touch config.yml

  7. Open your editor of choice (e.g. Notepad++) and edit config.yml (be sure to use Unix line endings!)—also make sure the drupal_deploy_repo varaible is set to the correct GitHub repository you want to use as your origin remote:

    ---
    vagrant_synced_folders: []
    vagrant_synced_folder_default_type: ""
    
    vagrant_hostname: local.projectname.com
    vagrant_machine_name: projectname
    vagrant_ip: 192.168.88.25
    
    drupal_deploy: true
    drupal_deploy_repo: "[email protected]:github-username/projectname.git"
    drupal_deploy_version: master
    drupal_deploy_update: true
    drupal_deploy_dir: "/var/www/projectname"
    drupal_deploy_accept_hostkey: yes
    drupal_core_path: "{{ drupal_deploy_dir }}/docroot"
    ssh_home: "{{ drupal_deploy_dir }}"
    
    drupal_build_composer: false
    drupal_composer_path: false
    drupal_build_composer_project: false
    drupal_install_site: false
    
    firewall_allowed_tcp_ports:
      - "22"
      - "25"
      - "80"
      - "81"
      - "443"
      - "4444"
      - "8025"
      - "8080"
      - "8443"
      - "8983"
      - "9200"
      # For reverse-mount Samba share.
      - "137"
      - "138"
      - "139"
      - "445"
    
    # Run the setup script.
    pre_provision_scripts:
      - "../examples/scripts/setup.sh"
    
    # BLT-specific overrides.
    vagrant_box: geerlingguy/ubuntu1404
    
    drupal_db_user: drupal
    drupal_db_password: drupal
    drupal_db_name: drupal
    
    configure_drush_aliases: false
    
    # Use PHP 5.6.
    php_version: "5.6"
    php_packages_extra:
      - "php{{ php_version }}-bz2"
      - "php{{ php_version }}-imagick"
      - imagemagick
    
    nodejs_version: "4.x"
    nodejs_npm_global_packages:
      - name: bower
      - name: gulp-cli
    drupalvm_user: vagrant
    nodejs_install_npm_user: "{{ drupalvm_user }}"
    npm_config_prefix: "/home/{{ drupalvm_user }}/.npm-global"
    installed_extras:
      - adminer
      - drupalconsole
      - drush
      - mailhog
      - nodejs
      - selenium
      - xdebug
    
    # XDebug configuration.
    # Change this value to 1 in order to enable xdebug by default.
    php_xdebug_default_enable: 0
    php_xdebug_coverage_enable: 0
    # Change this value to 1 in order to enable xdebug on the cli.
    php_xdebug_cli_enable: 0
    php_xdebug_remote_enable: 1
    php_xdebug_remote_connect_back: 1
    # Use PHPSTORM for PHPStorm, sublime.xdebug for Sublime Text.
    php_xdebug_idekey: PHPSTORM
    php_xdebug_max_nesting_level: 256
    php_xdebug_remote_port: "9000"
    
  8. Back in Cmder, run vagrant up

  9. Wait for Drupal VM to complete its initial provisioning. If you get an error, try running vagrant provision again.

Set up BLT inside the VM

BLT local refresh command in Cmder in Windows 7

  1. Run vagrant ssh to log into the VM.
  2. Make sure you're in the project root directory (e.g. cd /var/www/projectname).
  3. Run composer install to make sure all project dependencies are installed.
  4. Create blt/project.local.yml with the following contents:

    drush:
      aliases:
        local: self
    
  5. Run the command on this line to set up the blt alias inside the VM.

    • Note: After this BLT bug is fixed and your project is running a version of BLT with the fix included, you can just run: ./vendor/acquia/blt/scripts/drupal-vm/post-provision.sh
  6. Type exit to exit Vagrant, then vagrant ssh to log back in.
  7. Make sure you're back in the project root directory.
  8. Correct NPM permissions: sudo chown -R $USER:$USER ~/.npm-global
  9. Run blt local:refresh to pull down the database and run setup tasks.
    • Note that the first time you connect, you'll need to type yes when it asks if you want to accept the staging site's host key.

At this point, you can use the site locally at http://local.projectname.com/, and manage the codebase inside the VM.

Use PhpStorm to work on the codebase

For speed and compatibility reasons, this method of using Drupal VM to run a BLT project is done 'outside in', where everything is done inside the VM. But if you want to edit the codebase in a native Windows-based editor or IDE, you need access to the project files.

PhpStorm is a fully-featured PHP development IDE from Jetbrains, and is used by many in the Drupal community due to its speed and deep integration with PHP projects and Drupal in particular. You can download a free trial, or acquire a license for PhpStorm to use it.

One benefit of using PhpStorm is that it can work directly with the project codebase inside Drupal VM, so you don’t need to configure a shared folder.

  1. Open PhpStorm.
  2. Click “Create New Project from Existing Files” (if you are in a project already, choose File > “New Project from Existing Files…”).
  3. Choose the option “Web server is on remote host, files are accessible via FTP/SFTP/FTPS.” and then click “Next”.
  4. For “Project name”, enter Projectname
  5. Leave everything else as default, and click “Next”.
  6. In the ‘Add Remote Server’ step, add the following information:
    1. Name: Drupal VM
    2. Type: SFTP
    3. SFTP host: local.projectname.com
    4. Port: 22
    5. Root path: /var/www/projectname
    6. User name: vagrant
    7. Auth type: Password
    8. Password: vagrant
    9. Click ‘Test SFTP connection’ and accept the host key if prompted.
    10. Ensure the “Web server root URL” is correct (should be “http://local.projectname.com”).
  7. Click “Next”
  8. Click on the “Project Root” item in the “Choose Remote Path” dialog, and click “Next”.
    1. Leave the ‘Web path’ empty, and click “Finish”.
  9. PhpStorm will start ‘Collecting files’ (this could take a couple minutes).
  10. Change the Deployment configuration in the menu option Tools > Deployment > “Configuration…”
  11. Click on ‘Excluded Paths’ and add deployment paths containing things like Composer and Node.js dependencies:
    1. “/vendor”
    2. “/docroot/themes/custom/project_theme/node_modules”
  12. Enable automatic uploads in menu option Tools > Deployment > “Automatic Upload”

From this point on, you should be able to manage the codebase within PhpStorm. You can also open an SSH session inside the VM directly from PhpStorm—just go to the menu option Tools > “Start SSH Session…”

PhpStorm might also ask if this is a Drupal project—if so you can click on the option to enable it, and make sure the Drupal version is set to the proper version for your site.

Note: If you perform git operations on the codebase running inside the VM, or other operations like configuration exports which generate new files or update existing files, you need to manually sync files back to PhpStorm (e.g. click on project folder and click menu option Tools > Deployment > "Download from Drupal VM". At this time, only automatic upload of files you edit within PhpStorm is supported. See the issue Auto Refresh of Remote Files for more info and progress towards an automated solution.

Note 2: If you encounter issues with CSS and JS aggregation, or things like Stage File Proxy showing logged errors like Stage File Proxy encountered an unknown error by retrieving file [filename], you might need to manually create the public files folder inside the project docroot (e.g. inside Drupal VM, run mkdir files wherever the public files directory should be.

Configure a reverse-mounted shared folder

Add a Network Location to Windows 7 from Drupal VM's Samba share

If you don't have a PhpStorm license, or prefer another editor or IDE that doesn't allow working on remote codebases (via SFTP), then you can manually create a Samba share instead. In this case, we're going to use Samba, which requires the TCP ports listed in the customized config.yml file above to be open (137, 138, 139, and 445).

  1. (Inside the VM) Install Samba: sudo apt-get install samba
  2. Add the following contents to the bottom of /etc/samba/smb.conf:

    [projectname-share]
       comment = projectname
       path = /var/www/projectname
       guest ok = yes
       force user = vagrant
       browseable = yes
       read only = no
       writeable = yes
       create mask = 0777
       directory mask = 0777
       force create mode = 777
       force directory mode = 777
       force security mode = 777
       force directory security mode = 777
    
  3. Restart the Samba daemon: sudo service smbd restart (sudo systemctl restart smbd.service on systems with systemd).

  4. Set Samba to start on boot: sudo update-rc.d samba defaults (sudo systemctl enable smbd.service on systems with systemd).
  5. (Back on Windows) Mount the shared folder from Windows Explorer: \\local.projectname.com\projectname-share
  6. You can now browse files from within Windows just as you would any other folder.
  7. If you'd like, you can 'Map a new network drive' for more convenient access:
    1. Right-click while browsing 'Computer' and select "Add a network location"
    2. Go through the wizard, and add the shared folder location you used above (\\local.projectname.com\projectname-share).
    3. Save the new network location with a descriptive name (e.g. "BEAGOV").
    4. Now, you can just go to Computer and open the network location directly.

Notes:

  • You can only access the network location when the VM is running. If it is offline, files are inaccessible.
  • You should always use Unix line endings in your editor to prevent strange errors.
  • You can set guest ok = no if you want to make sure the shared directory is protected by a login. However, unless you change the Drupal VM defaults, the share should only be accessible internally on your computer (and not on the wider LAN).
  • You can also open a folder as a workspace, for example in Notepad++, by manually entering the network path (\\local.projectname.com\projectname-share).
  • Global operations (e.g. project-wide search and replace) are going to be very slow if done on the Windows side. It's better to do that either in the VM itself (using tools like grep or vim), or on another computer that can work on the files locally!

Conclusion

Using this 'outside in' approach makes it so you can get near-native performance when building Drupal sites locally on Windows 7, 8, or 10. It requires a little extra effort to get it working correctly, and there are still a couple small tradeoffs, but it's a lot faster and easier to set up than if you were to use normal shared folders!

Hopefully the SSHFS issues I mentioned earlier are fixed soon, so that the reverse shared folder will be easier to configure (and not require any manual steps inside the VM!).

Also, note that you could use the vagrant-exec plugin to run commands inside the VM without first logging in via vagrant ssh.

Comments

This didn't work at first for me with my VM that I've been working in. Once I completely disabled the firewall in the VM it worked fine, though. This is by far the best way for Windows users to work on a BLT project.

To get even more speed on windows I use the "Tools->Deployment" options of PHPStorm.
I have a local clone of the repo on my normal filesystem, which I work on. This makes search/replace/edit native fast.
Any and all changes are automatically (enable through options) synced to either a samba share (either from a VM or from a remote server) or via SFTP (again, to a VM or a remote server).

https://www.jetbrains.com/help/phpstorm/2017.1/deployment.html

@JvE - very helpful tip! I think some other IDEs have similar options, too, like Coda. Also, if you use an app like Transmit (Mac-only, but there are similar apps for Windows), you can 'mount' a folder like a disk just using SSH.

Reverse Samba share is the most performing solution for now !
Drupal vm is certainly the best solution for Drupal ready virtual lamp server.
Thank you Jeff !

Hello.
Thank you for a great article and DrupalVM.
I`m currently using DrupalVM on windows machine (Surface Pro 3, i5, 8GB RAM) in a bit different setup.
I tried a lot of different solutions to increase I/O performance coz it is obviously the weakest part on windows machine and found that the vagrant plugin "vagrant-winnfsd" works fast and stable for me.
I take default DrupalVM configuration and add only one line to Vagrantfile:
config.vm.network "private_network", type: "dhcp"

Then run vagrant up under administrator (actually, I`m running PHPStorm under administrator to avoid file access issues).
No issues so far except the git under vagrant machine (we have custom composer bash scripts on our projects so it should be called under guest machine). Git is a bit slow in this setup.

Also, I tried to use http://docksal.io/ which is based on docker and it works fast too, but the setup of the new project is slower then DrupalVM and I miss useful tools like mailhog, adminer, etc.

Hope that SSHFS issues will be fixed soon and I`ll definitely try it and check performance of that setup.

Hi Jeff,

probably I'm asking a stupid question, but what needs to be inside the git repository which you specified in the config.yml file?
I've tried a lot of different approaches but none of them worked.
It would be great if you could give me some additional help on this.

Thank you

Am I correct in (like the poster above) assuming that this REQUIRES you to have a BLT project in a repository already to get your BLT project running on the Drupal VM?

I can get this to work except my repo is empty - I am just starting the project.

Is there an alternative path to these instructions where the BLT project could be created from scratch entirely on the Drupal VM?

I had to use
path = /var/www/drupalvm
rather than
path = /var/www/projectname

Hi Jeff,

I am a newbie to the world of Drupal and VM and blt. I am trying to get an existing drupal project running on my local Windows 10 machine. I am running into some issues one is that I cannot get blt to alias.

When I run $ composer info acquia/blt
name : acquia/blt
descrip. : BLT
keywords : automation, deployment, drupal, template, testing
versions : * 9.2.8

When I run $ composer run-script blt-alias
> blt blt:init:shell-alias -y --ansi
The system cannot find the path specified.
[warning] Drupal VM is locally initialized, but is not running.
[warning] Could not find your CLI configuration file.
[warning] Looked in ~/.zsh, ~/.bash_profile, ~/.bashrc, ~/.profile, and ~/.functions.
[warning] Please create one of the aforementioned files, or create the BLT alias manually.

I cannot figure out how to create the BLT alias manually.

Any help appreciated.