Migrating 20,000 images, audio clips, and video clips into Drupal 8

tl;dr: If you want to skip the 'how-to' part and explanation, check out the pix_migrate example Drupal 8 migration module on GitHub.

For a couple years, I wanted to work on my first personal site migration into Drupal 8, for the last Drupal 6 site I had running on my servers. I've run a family photo/audio/video sharing website since 2009, and through the years it has accumulated hundreds of galleries, and over 20,000 media items.

Family Photos and Events website display - desktop and mobile
The home page of the Drupal 8 photo sharing website.

Migrating Link fields with multiple properties with Drupal 8

Today I needed to migrate a URL/Link into a Drupal 8 site, and I was scratching my head over how to migrate it so there were distinct values for the URL (the actual link) and the Label (the 'title' that displays to end users and is clickable). Drupal 8's Link field type allows you to set a URL in addition to an optional (or required) label, but by default, if you just migrate the URL, the label will be blank.

I first set up the migration config like so:

  field_url: source_url

And source_url was defined in the migration's source.fields configuration.

In my case, the source data didn't have a label, but I wanted to set a default label so the Drupal 8 site could display that as the clickable link (instead of an ugly long URL). To do that, it's similar to migrating a formatted text field, where you can migrate individual components of the field using the syntax [field_name]/[component]. In a Link field's case, it looks like:

Poor Man's XHProf profiling of Drupal 8 Migrations and Drush commands

On a recent project, there was a Migration run that took a very long time, and I couldn't pinpoint why; there were multiple migrations, and none of the others took very long at all (usually processing at least hundreds if not thousands of nodes per minute). In Drupal 7, if you enabled the XHProf module, then you'd get a checkbox on the configuration page that would turn on profiling for all page requests and Drush commands.

In Drupal 8, the XHProf module was completely rewritten, and as a side effect, the Drush/CLI profiling functionality is not yet present (see: Profile drush/CLI with XHProf in Drupal 8).

Since I don't have the time right now to help figure out how to get things working through the official XHProf module, I decided to use a 'poor man's profiling' method to profile a Migration run:

Migrate a custom JSON feed in Drupal 8 with Migrate Source JSON

June 2016 Update: Times change fast! Already, the migrate_source_json module mentioned in the post has been (mostly) merged directly into the migrate_plus module, so if you're building a new migration now, you should use the migrate_plus JSON plugin if at all possible. See Mike Ryan's blog post Drupal 8 plugins for XML and JSON migrations for more info.

Recently I needed to migrate a small set of content into a Drupal 8 site from a JSON feed, and since documentation for this particular scenario is slightly thin, I decided I'd post the entire process here.

I was given a JSON feed available over the public URL http://www.example.com/api/products.json which looked something like:

Migrating style and script tags from node bodies into Code per Node

For a recent project, I needed to migrate anything inside <script> and <style> tags that were embedded with other content inside the body field of Drupal 6 nodes into separate Code per Node-provided fields for Javascript and CSS. (Code per Node is a handy module that lets content authors easily manage CSS/JS per node/block, and saves the styles and scripts to the filesystem for inclusion when the node is rendered—read more about CPN goodness here).

The key is to get all the styles and scripts into a string (separately), then pass that data into an array in the format:

Migrating Drupal 7 users from site to site while preserving password hashes

From time to time, I use the incredibly powerful Migrate module to migrate a subset of users from one Drupal 7 site to another.

Setting up the user migration class is pretty straightforward, and there are some great examples out there for the overall process. However, I couldn't find any particular documentation for how to preserve user passwords when migrating users from D7 to D7. It's simple enough to set the 'md5_passwords' boolean for Drupal 6 to Drupal 7 user migrations, so passwords will be updated when a user logs in the first time on the D7 site... but it's not as straightforward if you want to simply move the salted/hashed passwords from D7 to D7.

During the migration, when the user account is saved, Drupal will re-salt and re-hash the already-hashed-and-salted password you pass in through your field mappings, and users will have to reset their passwords to log in again.

To override this behavior, you need to implement the complete() function in your user migration, and manually overwrite the just-saved user account password field:

Using Migrate to Import Content from a Legacy Database

Since a few people who missed last night's St. Louis Drupal meetup wanted to hear more about Migrate module and my notes from a recent content migration for flockNote, I thought I'd post my observations and some tips here.

Migrate Module, v2

My prior experience with Migrate was on version 1.x, along with Table Wizard, for Drupal 6 (I used it in tandem with a bunch of CSV files that were used to import organizational data into the Archdiocese of St. Louis' website). A lot has changed in the process of Migrate upgrading from 1.x to 2.x... especially with Drupal 7!

Subscribe to RSS - migrate