Did breaking backwards compatibility kill Drupal?

First of all, Drupal is not dead. But I would argue it's not in healthy place relative to competing projects as it was in its heyday, in the early 2010s.

In this blog post, I will explore the problem the Drupal community finds itself in five years after a major release that broke backwards compatibility in almost every subsystem, forcing a laborious upgrade process and process shift that left many users in the dust.

I've written about this in the past, most famously in my post Drupal 8 successes and failures. I'm not going to rehash the details from that post, but I did want to focus on what I think is the primary reason for this graph's downward trajectory since 2016:

Usage Statistics for Drupal Core from 2013 to 2020

Unlike past releases like Drupal 5, 6, and 7, the release of Drupal 8 did not result in many 7-to-8 upgrades and new Drupal sites. Rather, there began a gradual decline in Drupal 7 sites, along with a very low rate of new Drupal 8 sites to replace them.

Looking at the usage graph for the years prior to Drupal 8's release, you can see Drupal's overall usage doubled from 2011 to 2014:

Usage Statistics for Drupal Core from 2010 to 2014

For the first time since 2014, the Drupal project is likely to see less than one million active installations reporting back in to Drupal.org later this year.

Note: These graphs do not show every single Drupal site that's running in production. But they are the best approximation and on relative time scales, and they show relevant trends, since in Drupal 7, 8, and 9, the 'Update Status' module (which is key to this data on Drupal.org) is enabled in the standard installation.

Breaking Backwards Compatibility

The main motivation for this blog post was the constant 'yin and yang' in the software industry for encouraging radical design departure and adoption of new paradigms vs. the pragmatic "we will never break your code" approach of something like Microsoft Windows.

Each has its ups and downs, and each has wildly successful—and wildly disastrous—examples.

  • Radical departure: Apple has reinvented its OS and application ecosystem multiple times, switching entire platforms (68k to PPC to Intel and now to ARM) and programming languages (Pascal, C/C++, Objective C, Swift) along the way. Some developers drop off with each shift, but Apple has now built one of the most profitable app ecosystems and is the most valuable corporation in the world by market cap.
  • We'll never break your code: PHP itself seems to be a shining example. With the exception of small changes every decade or so to fix new fatal errors, I have many scripts and libraries written for PHP 5.0 which work unmodified on PHP 7.4 and PHP 8!

When I was reading the post Dear Google Cloud: Your Deprecation Policy is Killing You by Steve Yegge, I think I realized why some projects are successful one way and some are successful the other:

Consistency in deprecation policies.

Drupal, since its inception, was an 'island'. It was written in PHP, but generally there weren't efforts to make Drupal a part of the growing PHP ecosystem, like newer, framework-oriented projects like Symfony or Laravel. It stayed this way for over a decade.

A similar case is the Python language. It is a full-featured programming language, with solid and reliable core APIs. CS graduates were taught one of Java or Python (or both), and Python-based courses, books, and software basically worked the same for almost ten years.

But in both cases, a major change decimated the implicit trust developers had built around the project over a decade: an unspoken promise the project wouldn't make sweeping changes that require major rewrites or rearchitecture in one major version.

In Drupal's case, a major effort was made in Drupal 8 to "get off the island", and replace many core APIs and systems with Symfony and other PHP ecosystem libraries. It was an ambitious effort, and Drupal today is a pretty well-architected PHP application.

But it broke the implicit trust developers built over many prior releases.

The upgrade process was no longer a case of "a few hours or a few weeks," but instead "a few months," since in many cases upgrading meant a full redesign, and in many cases, complete site rearchitecture.

In Python's case, Python 3 made breaking changes which were not adopted by many popular libraries and frameworks for many years, and it took twelve years before the community deemed it safe to once and for all drop Python 2 support.

What is the takeaway?

Well, on the positive side, the Drupal community seems to be learning from the massive misstep in Drupal 8—Drupal 9, which was just released earlier this year, has no major refactorings, only removals of old, deprecated APIs. If you have a site that works in the latest version of Drupal 8, it's highly likely the upgrade process will take minutes, or at most a few hours.

And Drupal 10 seems to be on a similar path.

But, as someone who spent ten years earning the majority of my income from Drupal development, and is still a somewhat invested member of the Drupal open source community, I am left wondering where to go from here.

Many of the features of Drupal 8, 9, and now 10, seem to be focused on high-dollar enterprise projects competing with multi-million dollar Adobe Experience Manager or Sitecore projects, as the lower ('less profitable') end of the CMS market has been abandoned or left to Wordpress and online site builders like Wix or Squarespace.

But that's not the thing that worries me most.

I don't think it's a stretch to say developers like me have a sort of 'PTSD' after a foundational shift in the trend of backwards compatibility breaks, and I personally feel less enthusiastic about supporting open source code that I fear may need more major rewrites just to 'keep up with the Joneses'.

If you build in an ecosystem that often breaks backwards compatibility (Apple's OSes, Google Cloud, and especially most of the Javascript ecosystem), then you are conditioned to expect it, and that's part of the 'cost of living' in that ecosystem.

But if you work with an ecosystem that is stable for long periods, then makes multiple major BC breaks in one release, it feels like you're getting hoodwinked.

Conclusion

Take the real-world experience of the Drupal community to heart: figure out if your project is more like an Apple or more like a Microsoft.

Do you have a reputation of breaking backwards compatibility frequently? Or do you typically make sure you have BC layers that allow people using your software to be more lazy when it comes to upgrades?

Neither is necessarily wrong. But making a radical departure in one major release may harm your project in the long term.

Comments

I loved PHP as a procedural language. It wasn't that Drupal adopted symfony that chased me away, it was that Drupal changed to an object-oriented design, which didn't interest me at all. So I flushed 15 years of Drupal behind me and made a career change. I'm still sad for what Drupal did to itself. PHP isn't a language people pick up as a college course to do professional programming - it's a hobby language and they switched to APIs which alienated us hobbiests in the interest of trying to become a platform businesses would "take seriously", but all I keep doing is point at WordPress to counter that argument. Long live Backdrop!

- user 972 on Drupal.org

Going to OO was not a Drupal decision. Going to Symphony was what created the necessity to do OO. All frameworks that I know of are OO. Why is that? because OO is a better design approach. Just because "some" people don't understand it doesn't take that away. Its like saying I don't understand climate science and therefore climate change is not real. That's a head in the sand approach.

Despite the Apple changes, they make strenuous efforts to keep existing software working; fat binaries have been a thing since the 1990's, with their shift from 68K architecture to PowerPC. And they have kept the same underlying idea even in the current system. They kept parallel OS systems going even as they reworked the OS from the ground up. So they may make radical changes, but they give developers the chance to travel with them.

Crossing the Chasm is probably the seminal book about growing a tech product/company. It posits that there are points in the growth where there's a before and an after, and that you lose users as you cross that chasm. I still work on Drupal, but my knowledge stretching back to version 4.x is relatively useless with 8 and beyond. I think that Drupal got it terribly wrong - that you may change user base, but that you have to take developers with you if you want to survive. Apple understand that. Drupal didn't. Even more than this, for sustainability you have to have an entry mechanism for developers. Drupal has killed the entry point. Hobbyists do not turn to Drupal.

I'd go much further and argue that Drupal also misread the audience, despite many of us arguing against it. Growth comes from the bottom of the market, not the top, and Drupal decided cede the bottom end. Crossing the Chasm deals with this extensively, showing that companies that give up the entry market end up in a niche that they can't escape, a niche which becomes more and more irrelevant.

In a world where Drupal didn't give up on the pre-8 architecture, we'd have subsytems to allow D7 modules to run on D8 with a few minimal changes. And if we were competing with the low-end, we'd take UI seriously, and would have at least a distro that Just Worked, with a slick interface. It would not take an engineer to install or maintain Drupal.

For me, it wasn't specifically the shift to OOP that is a problem, it is that now you often need to know / remember more things to achieve the same outcome and a lot of my previous experience with drupal prior to 8 is of little use anymore.
Take for example a couple of days ago when I wanted to create a custom entity. In drupal 7 it was fairly easy, implement a couple of hooks returning arrays. Now, with Drupal 8/9, I have a multitude of files, with yml, php and annotations for which I can never seem to find helpful documentation on.
The only real thing that has kept me in the Drupal world after working with it for over 10 years is the fact I can still get work using it thanks to the Australian government adopting it. As soon as that work runs out, I will probably be jumping ship.

Creating a new (content) entity type Drupal 8/9/10: `drush gen content-entity`

Implementing a couple of hooks returning arrays for creating a custom entity would give you a half functional entity type (access handler? fieldable? revisions and moderations? views integration? rest?). It was ok 10 years ago, but I can't see that being a competitive feature with nowadays requirements.
Drupal is now more complex, not complicated. Like many other things. Go back 15-20 years ago and talk to your younger self about building a Javascript application consuming REST API data, and after the WTFs show her/him Reacts code to get even more WTFs.

The world is getting more complex and as the post says you can decide to stay more conservative or radical, but none of them is wrong. It's just different. Long Live Difference. If you drop something that's on you, not on the product you've dropped.

I think an underlooked reason for the big level of change in Drupal8 is underlooked - the looong development cycle.

This had been happening before and the Drupal 7 development cycle wasnt short either, but the longer the development cycle the more breaking changes that get piled up on each other.

I hope we continue with the shorter (both major and minor) releases cycles that seem to become necessary to follow Symfony.

Breaking of conventions and "verticalization" killed Drupal. Drupal 8 to 9 switch still contributed to the first, because it erased some existing procedural conventions. Sweeping verticalization under the roof of a few large companies and ideologizing Drupal, turning it into a wokeness club, ideology policing, and banning people in Slack contributed to the second.

It is true that Drupal caters towards the high end od the market but it is do flexible that you can build low cost websites with it. Check out Droopler distribution which will help you do just that.

As long as it relies on composer to do something as simple as use the addressfield module, it will be out of the reach of most hobbyists and those people that dipped a toe in the water with drupal, and progressed on to be full contributors.

I'm fairly experienced with Drupal 8 and the composer world now, have a graduate degree in CS from a major university, and work as a developer using php, java, and python for decades and I STILL find myself in composer black holes I have to spend considerable time researching to fix/correct.

No casual user or hobbyist will do that-- period.

And anyone telling themselves they will is hallucinating.

Friends, let me also express my point of view.
As soon as Drupal 8 came along, I can no longer do anything on the 7th and 6th projects that I have worked with for over 10 years.
Hooks are a nightmare, arrays are a nightmare, everywhere you need to memorize a lot and parse a lot of code, it's easy to make mistakes.
What can not be said in OOP, but you need to rebuild how to switch from templates to twig, but this then pays off many times over.
I can recommend studying the book by Deni Sipos on the development of Drupal modules and Lockhart - Modern php, I think, with experience in previous versions, the entry threshold should not be high than for beginners.

"Many of the features of Drupal 8, 9, and now 10, seem to be focused on high-dollar enterprise projects"

This is a common statement. It would be great to back this up with supporting examples though. Dries announced at this recent DrupalCon Global keynote that Automated updates, Easy out of the box (Media and Layout Builder as default experiences) are part of a shortlist of initiatives. The new Olivero frontend theme is also one of the five and is not something any enterprise would use either. I would argue most things proposed for Drupal 10 (as in later releases of Drupal 9) are features that benefit sites of any size and in some cases especially small sites. Enterprise builders can figure out how to make good use of Media or Layout builder regardless whether they are default or not and they will definitely not use the default Drupal theme. They would not care if there would not even be one.

I would not argue with most of the other parts of your post as it raises really good points, but this is heavily in need of supporting facts.

Composer. The switch to composer has been very difficult for low budget agencies and non-technical site users. What used to be a matter running a simple drush command or extracting a tar.gz file into the sites/all/modules directory has become a nightmare of dealing with composer's quirks and steep learning curve. The fact that it was a later addition to Drupal 8 didn't help things either.

I'm (mostly) used to it by now, but I'm still astounded at how much easier it us to install and update modules in Drupal 7 compared to Drupal 8/9.

I fully agree with composer being a bottleneck. That is one single thing and not a "many features". The "many features" position depicts Drupal as fundamentally abandoning site builder features that benefit small to big, which is just not the case. Depending on how we look at Drupal, the solution will be different, which is why I point this out. Eg. "radically simplify the composer workflow" and "add automated updates" are features we can add and not fundamental mind-shifts, because layouts, media, etc. that were worked on benefit everyone from small to big already.

I completely agree. Add Docker (DDEV) to the "new landscape" and its just overwhelming for someone who doesn't live in Drupal every day.

I'm going to paste my comment from HN. Hope that's ok.

I don't believe that it was purely because Drupal 8 broke compatibility. Many were excited about Drupal 8.
Drupal fell behind in some areas. Looking at Wordpress and Gutenberg (while not perfect) it has leapt forward to provide a modern content editing experience. Perhaps Acquia focused on the wrong things? Perhaps other companies making money on Drupal didn't invest enough to improve it?

Drupal.org needs to be a flagship of what Drupal can be. The documentation is not well organised. Page loads are slow. Contributing can be difficult for newcomers. Other open source projects are better at selling themselves through their project website, documentation and code examples. Drupal.org shows its age.

Drupal can take a lot of time to setup and theme. Keeping Drupal installations up to date requires expertise. Smaller clients these days realise Drupal might be too expensive and found an alternative CMS or moved to a Saas. Laravel probably also took a big chunk of custom development away from Drupal.

At some point I think the community can only do so much.

There is nostalgia. Many times I have thought to myself: If only I could give all my time to a project like Drupal. That would make me happy. I know, I know.. I could have. I should have... For me, life got busy and my career went in a different direction.

I also think the situation expelling Crell was weird. It felt like bridges got burned. I know people I've met who also felt it was a strange way to handle the situation. Can't we make that right?

This is not to criticise anyone. It is just my opinion. I am not a big time Drupaler, so you could see this as an outside perspective.

Drupal.org is a good bellwether for the upgrade/migration pain. I remember it took like 3-4 years after Drupal 7's release before Drupal.org was upgrade.

We're 5 years past Drupal 8's release, and now Drupal 9 is here, and Drupal.org still hasn't begun an upgrade from Drupal 7 in earnest. It's a huge effort, and I think it's indicative of the amount of pain and planning that others have to go through to consider a 7-to-8/9 upgrade currently. There was a statement a few months ago that "Migrations are complete for Drupal 8!", but it felt a little like "Mission Accomplished" in the US a decade ago, where we celebrated a random milestone, but the ground truth was much different.

There are still weird migration bugs that I've run into in the past few months upgrading very simple Drupal 7 sites (with few contributed modules, all of which were available in Drupal 8) that were not documented, and it took way too long to figure out what the problem was.

I'm petty sure Drupal can achieve the same editorial experience as wordpress using components as glazed builder. Give a try!

Drupal still has lots of outdated sub-systems that are almost impossible to replace/refactor because the current BC policies.

"This is a common statement. It would be great to back this up with supporting examples though. Dries announced at this recent DrupalCon Global keynote that Automated updates, Easy out of the box (Media and Layout Builder as default experiences) are part of a shortlist of initiatives"

I would argue that the proof you seek is in your own statement and the current timeline. Dries announced those as shortlist initiatives... five /years/ after Drupal 8 released. It has taken this long to begin these initiatives, and my own limited experience with the early versions of the features leads me to believe they aren't the most intuitive of things (thus I assume the initiatives to improve them). I'm not sure I can point to positive proof that "Many of the features of Drupal 8, 9, and now 10, seem to be focused on high-dollar enterprise projects" but I can point to the lack of low tier features and ease of use, because I'm pretty sure no one would say that Drupal is "easy" compared to other low end solutions (Wordpress, Wix, etc.).

Bear with me here, but... what if Drupal had just been stopped at 7 and that was that?

Developers could have moved on to wherever they liked, taking the lessons of Drupal's successes and failures. Some could have unofficially carried on (like Backdrop), companies would have stepped in to provide paid security fixes.

As discussed, migrating a site of any complexity from 7 to 8 was going to be a rebuild, so nothing would be lost there. As Drupal 7 was simpler, code-wise, than 8, and many of its paradigms were now outdated, there was some, but not catastrophic loss there and developers were having to retrain on 8 anyway.

There would have been no half-decade long struggle to upgrade parts of a complex system and still receive negative feedback about it and a decrease in usage, and those talented developers could have put their efforts elsewhere. Some may have started new projects that were cleaner from the beginning (and 15 years in the future would have become what Drupal was - that's how it goes) and didn't have the baggage. Site builders similarly would have had options.

Drupal is now "focused on high-dollar enterprise projects competing with multi-million dollar" but still with baggage from it's roots, meaning it's often not fun for developers, and the path to competing for those projects isn't smooth.

(I understand why this didn't happen, I'm just saying what if it did? What if projects consciously took this choice sometimes? Also, I've worked on many D8 sites - I didn't jump ship.)

This is the path I'm on. Working with D7 everyday writing code for it at my day job. The only thing I wish I had is big pipe; the ability to render slow parts of the page outside of the first html load. I did some experiments with edge side includes but I was fighting core every step of the way so I never moved past R&D in d7.

On the other hand I have asynchronous MySQL (apdqc), instant workers allowing for parallel code execution (httprl), and non blocking rendering of the page (advagg). We're 100% logged in traffic, only using MySQL for cache tables and the site hasn't died due to database issues and it's over 200gb in size with heavy traffic (multiple web heads). For us d7 still gets the job done, and I only see downsides if we upgrade to d8 as I'd have to spend a lot of time just getting apdqc and httprl converted over as those are lifesavers for us. Seriously httprl_call_user_func_array_async() is a game changer as is cache prefetching and async writing for all cache tables as well as a selection of other tables. The site is just so much faster with these changes. Luckily advagg did get enough interest to get a d8 port but I'm not sure if you can get a perfect score like you can get with d7; see my Nashville presentation to see it.

Mikey your approach is great, I'm petty sure there is a lot of new community comers willing to help with the migration of the apdqc. Maybe the Redis and other performance D8 focus people are in the same track.

Lack of backwards compatibility certainly burned a LOT of contrib developers who put their blood and sweat into their D7 modules. Then D8 comes out and says, hey now you have to reprogram everything.

The direction of building "ambitious" projects is nice, but I just want to build normal websites. D7 was so easy to use. D8 made things a lot harder and complex starting with requiring composer to install/manage it, so that most likely turned off LOTS of potential new users/sites.

I'm finding that it's easier to migrate to D6/7 site to Wordpress than it is to D8/9.

"I'm finding that it's easier to migrate to D6/7 site to Wordpress than it is to D8/9."

I think I'm going to look into this as well.

So... I mostly agree with the outlined premise here that backward compatibility issues is a major factor that has caused a decline in Drupal in recent years, although I entirely disagree with the details to support that -- and perhaps the other major factor of developer alienation is well represented in the comments in this thread.

Backwards compatibility issues introduced in Drupal 8? Which prior major Drupal release did not break backwards compatibility? I would argue that Drupal 8 was the first release of Drupal that ever promised to solve the backwards compatibility issue that Drupal always had in previous versions -- and represents the great maturing of the platform.

We had dozens of clients with Drupal 6 sites that never upgraded to Drupal 7, because the cost of migration was too high for the benefit gained. We've migrated many of these straight to Drupal 8, and some of those are on the verge of being Drupal 9 with no new upgrade project needed. The problem is not that suddenly it got much harder to upgrade to Drupal 8 compared to earlier versions -- the problem is that sites have gotten bigger, more complex, and harder to migrate. If we were trying to upgrade similar size sites from Drupal 5 or 6 to Drupal 7, it would be a bigger challenge than moving to 8 (and 9), mainly because the migration tools in 8 have gotten so much better. And we've finally reached the point where we're seeing more Drupal 8 than 7 -- these upgrade woes are nearing the end.

So, yes, Drupal's fatal flaw has always been major version upgrades -- up until Drupal 9, where suddenly it's no longer a flaw but a strength.

And, the reason this has led to declining Drupal usage is simply that end users, non-developers, are just plain tired of endlessly having to rebuild their entire site again, for minor gain, at similar (or usually greater) cost than it was to make in the first place. Nobody's happy at shelling out another 5 figures to build their site yet again every 4 - 5 years -- this has lead to a lot of businesses running away shrieking from Drupal.

The irony is, Drupal 8 and 9 solves this problem, but too late to stop a mass exodus from the platform.

That's only one half of the problem. The other half is Drupal developers. There are plenty of great Drupal developers out there, and I don't mean to cast shade on anyone -- but the simple fact is there's a large contingent of Drupal "developers" who appear to be scared of "modern" architectures like object-oriented programming and dependency injection. I quote "developers" because this is basic programming patterns that have been popular for 30 - 40 years now, hardly cutting edge stuff. I think that many Drupal developers hacked their way into the business by cutting and pasting code, picking up things as they went along, not through any formal training. And they may have an inferiority complex when they look at some big OO Java application and the devs who love that sort of thing. Hell, that description fits me as much as anyone -- I have a liberal arts degree, and the only programming class I took was back in middle school decades ago before the web was a thing.

The thing is, the web does keep growing and changing, and if you're not staying current, you're going to be left behind. And Drupal has done exactly that, maintaining a consistent mental model for site admins and builders while replacing the spaghetti obtuse unupgradeable hook system with a solid shiny powerful new engine under the hood, while adding support for all the shiny new things like turnkey APIs for headless web apps, powerful layout builders that don't destroy your information architecture, and solid media management tools.

If you're yet another English major with an inferiority complex about modern coding practice, well, I'll just say, give yourself some credit. You can do this -- you mastered the old wreck that was Drupal core, the new one is so much better, you just may need to learn some new tools. And this is the last major upgrade you'll need for at least another decade or two -- Drupal 8/9 is the best application platform out there, and well worth your time. Sure when things go wrong, they can be really tricky to figure out -- but these edges get smoothed out all the time, and if you can wield xdebug and a profiler, you can get to the bottom of it. You got this. We all got this. Drupal may have been taken a few notches down in popularity, but it's nothing but up from here.

I agree with this.
I feel pretty confused when I see comments from devs who say composer is so hard etc. I started building sites with D6, then D7 and now learning D8. I have never wrote a single line of PHP. I have been able to create my sites with existing modules, just theming a bit and CSS. So I am not a dev, but have been now happily using composer. So for me D8 does not look bad, actually yml is easier for me than PHP and composer works fine.

Not only do I agree with this post, I think it holds true for modern web development in general. Some 10-20 years ago, every enthousiast could build an impressive website quite easily. Drupal 7 and WordPress empowered self-proclaimed mediocre developers to compete. They enabled individuals - who wouldn't call themselves hardcore developers - to create stuff they could actually sell to professional organisations. Now it seems you need at least three specialists to create a "professional" modern website.

Drupal 8 seems to have steepened the already steep learning curve. Enter the success of WordPress, Gatsby, Eleventy, Contentful and no-code platforms: their strength lies in empowering a similar client-base as Drupal 7: the non-hardcore developer who wants to create something professional in less time. I would argue it is very dangerous to label them "non-enterprise" or "low-level", because eventually they will compete for the very "enterprise clients" Drupal 8, Adobe and Sitecore thought they had locked-in.

Adobe and site core? Are you joking? Have you had try to achieve at least the 50% of functionalities you can have with Drupal 8 in hours, in Adobe experience platform or Sitecore? Even in SharePoint... This is the real problem.. some people on Drupal community only have worked on Drupal and PHP. First look the competition bad points and compare please!

http://www.cmswire.com/information-management/10-lessons-from-hertzs-32…

Everything Freelock said was true. Migrations from d5 to 6 to 7 were all nightmares. Both from an end user and a developer side.

What I think worries me is that Drupal lost its open source pizazz. Think about the folks who made contributions in d7 both core and contrib. We've lost many smart people who've gone to the next cool thing like Javascript or go. This is nothing new; The biggest drupalcon was Portland 2013. The drupal association has no ability to build anything new without major outside investment of time and/or money. Contrib sprints are nearly non existent now. There is no road map or lead company like automattic with WordPress.

And to top it off there is a sense of drama in the community that makes some feel like contributing to drupal requires being in a special club.

You'd hope d8 to d9 would get more interest. Youd hope making targeted efforts to minorities would get more interest. But they haven't. The biggest contribs I see in the issue queues are folks trying to game the credit system by offering automated code sniffer patches to pad their resumes.

Contributor Braindrain from the many missteps of the past is IMHO the biggest existential threat to drupal.

Japerry, I too agree with Freelock and I share your concern about Drupal "losing its open source pizazz" and this being a very serious threat. I don't think the situation is as dark as you describe it but I certainly wish this new shine Drupal could create the drive and enthusiastic community that D6 and D7 enjoyed. Maybe it is something like PTSD and has to do with the adoption of new technologies as someone puts it above but it would say it feels like a coldness or as if some poison is in the air. My guess would be that the open source community spirit has at some point crashed on the fact that Drupal is managed by one or more persons or entities and not by a community/constitution/set of values.

The only thing that will "save" Drupal moving forward is leadership. Having someone at the helm who will not only listen to the user base, but implement the changes necessary to make it happen. Someone who can look at the new media library initiative and go: "Who are we making this for? Developers or people who need to USE Drupal?". I'm not a fan if you can't tell... We all just need a way to stick a stupid picture into a page without doing a song and dance to make it happen.

I briefly had the leadership conversation with someone more vested in the core of things and their comment was that Drupal is open source, therefore community led. I don't have a problem with open source. I don't have a problem with community led. I have a problem with 2000 captains of the ship trying to get their ideas implemented and it taking 10+ years for SOMEONE to make a decision.

WordPress works so well because a small group of people make the decisions and the community either helps make it better, or complains which prompts new decisions, which are immediately acted upon. SpaceX has blown up how many rockets and started over from scratch? And who's leading the commercial space industry? A company with a leader who is willing to make decisions, learn from them, then make it better. Drupal doesn't have this. If SpaceX was open source, we would watch 1000 engineers all trying to figure out the best way to build the first rocket model. Not the rocket, the model...

On a side note, I kicked off a small project on BackdropCMS the other day. I was joyously surprised. It felt like being at home again.

I really appreciate your perspectives, both this and the previous post. I don't necessarily disagree with any of the underlying principles or many of the comments, but I think we're missing some perspective.

Somethings happened in the industry outside of Drupal, you touched on it with the various SaaS products that have indeed gotten better. I actually lost a business association client that replaced their D7 site with a Wix site and it's absolute garbage now. But they can manage it entirely on their own. Couple that with the rise of JavaScript and many (most?) front end devs moving away from traditional CMS entirely. Drupal is facing more competition, as is PHP in general. I feel like a lot of these JavaScript frameworks are going to be biting customers in the ass in a few years.

WordPress likely still thrives because it's dead simple to set up as a backend content repository. Drupal does it better, but these new JavaScript devs don't want to be bothered with something complex like Drupal. Gutenberg is no panacea, many complaints from end users. In fact, my customers love using my Drupal sites compared to other systems (including WordPress), but I've put a lot of focus on the editorial experience. With Layout Builder and Media in core, Drupal can actually be so much better than the competition for building even small websites, which I mostly build.

WordPress is actually more comfortable with breaking changes since 5.0, and people are abandoning it as well. People hate change. Perhaps now is a golden opportunity for a Drupal resurgence?

I don't yet know how I feel about automatic updates. It's broken quite a few WordPress sites over the years. Not having code and config in Git is a risky proposition. But you must have someone (a dev) to manage git and deployment, etc. But many small businesses want to manage their own website without paying a dev, and there are plenty of low cost, no code SaaS options. Doesn't mean the product is good. Maybe we should explain the value of the professional site builder. This is missing from d.o.

Drupal has always broken between major versions. It could take months to migrate D6 to D7 on a large site. I don't think that was the biggest problem. It was a lot of change at one time. But the product is so much better now. We didn't explain that value. I use a fraction of the modules needed in D7 and have a much slicker product. If anything D8 has made me a better developer.

I feel like Drupal really suffers from poor marketing. Drupal.org is not convincing at best, most likely confusing, to new customers. Most people, if they've even heard of Drupal, have only heard how hard it is. We need a better message. I think that it starts with us. We need to be telling everyone how awesome Drupal actually is, and why their next project should be a Drupal project.

To be fair, there is a lot of things can be nice to speak open now. For example:. https://dxpr.com/. We need more like this if we want survive against other CMS, and js frameworks. Simply as can be: in the editorial experience does't improve, there is no chance for drupal in the future.

I love your approach:. "I feel like Drupal really suffers from poor marketing. Drupal.org is not convincing at best, most likely confusing, to new customers. Most people, if they've even heard of Drupal, have only heard how hard it is. We need a better message. I think that it starts with us. We need to be telling everyone how awesome Drupal actually is, and why their next project should be a Drupal project".

Yes you are right.

Here in my country Drupal is well know as the king's of the CMS on government over Oracle, SharePoint, Liferay. The only hard treats is Wordpress and the Js framework for their simplicity. In that ideas order, the community should be more focus on that layer: apps made with Drupal, Integrations, IA, Machine Learning, Blockchain, IOT..

Totally agree. All the technologies are moving fast, this is not only a Drupal thing. Maybe, in order to organize the mantenibility there should be a middle layer modules above core ones, maintained only by expert companies, and a third layer of modules mainteined by everyone. Maybe an other important matter is keep the custom modules as small as can be, in order to make less hard the upgrades, for example: The domain module: partitioned in smaller pices supported by different people. There are some large modules that are really tedius to migrate by just one person to D8.

Thank you for speaking my mind. I am a small drupal 7 site owner, feeling like drupal wants to force me to leave to other projects, for being small.

I used drupal because of my conviction that its the right platform for my needs. Thats seriously in question, to the extent that I have to start searching for alternatives to drupal for my project.

I don't believe that breaking code was the huge change that killed Drupal. It was an underline shift in philosophy and targeting enterprise that killed it. Symphony, composer, and the new way of coding killed it. There are many benefits for sure to the new system but simplicity is gone. Updates used to be a simple task with drush can now take hours to find what broke the update with composure. The same can be said for custom module creation. Things that were simple before now take to long. With so many simple frameworks out there it takes less time to create from scratch. The very thing a CMS was trying to solve is now more complicated than writing by hand.

Totally agree. I love Drupal 7 (8 years experience on Drupal) but I migrated almost all sites to WordPress. Most sites have medium difficulty.

While I don't necessarily disagree, it's strange to see backwards compatibility in Drupal 8 being blamed for the drop in sites. Drupal has always broken backwards compatibility between releases, but Drupal 8 was designed in a way that changes could be made without breaking backwards compatibility in the same way. Unfortunately this meant a long release cycle and a difficult upgrade from Drupal 7. In hindsight I would have liked to see a major release halfway between Drupal 7 and 8 so it wasn't such a big jump both for site upgrade complexity and developer skills.

I'd like to see some other stats too. If it is true that Drupal is now being used for more complex sites then there could be a larger number of developers building a smaller number of sites, being used by a larger number of consumers. I'm not convinced this is happening though, as I've also seen larger sites switching to WordPress. Are attendances at Drupal Con going up or down?

Here we are talking adrift.. maybe we can open an issue in drupal.org with all this thoughts. For sure we can address this as community.

It's not Python 3. It's Perl 6. The BC change is a symptom. Drupal 8 is a completely different system, technically, community wise, process wise -- everything. (With quite a bit of technical baggage still hauled from the past which doesn't help affairs.) Technically the biggest change is from implicit hacks to explicit boilerplate. To quote Values & Principles "focus shifted from writing the perfect code to growing the project, and amplifying Drupal's impact through better marketing, user experience, and more". The leadership model completely changed: instead of Dries ultimately arbitrating (and some lieutenants taking some work off his shoulders) we have product-framework-release managers and Dries has a visionary role. Instead of a communtiy of passionate developers who were passionate because Drupal mattered to their own projects we have people whose job is Drupal and if they do it right they release back to the community. Is it a surprise then you are less passionate for associating with your colleagues than your friends? Passion is actively punished, actually. And the number of Drupal sites decreasing is not the death of Drupal, it's just a new system and the numbers can't be compared, simply. Taller, not wider.

"The number of Drupal sites decreasing is not the death of Drupal, it's just a new system and the numbers can't be compared, simply. Taller, not wider."
Exactly.

I'd say it's a question of how "sexy" a new release is. Apple can afford to break backwards compatibility because their users enjoy the new, and don't depend much on old work. I'd say Drupals weak backwards compatibility would be easier to accept if new releases would be "sexy" where Drupal shines: in the ability to lego-like assemble complex data-structures in web pages. Drupal really is for the "builder" of relatively complex sites. While it still shines in that domain, it's too inconsistent and depends too much on 3rd party modules. I hope Drupal 10 will have a consistent and rich base for dealing with complex data-structures dan allow layouting them well. Currently Drupal is too much like Atom Editor while it should be like VSCode ;-)

Some people have mentioned this, but do people still build sites like they did in 2011 - 2014? Or has the whole paradigm of developing a "CMS" changed drastically. You can bicker within the community and long-time contributors about what went wrong in the project's philosophy, messaging, and planning; however, that doesn't matter to Joe Schmoe who is tasked to find the best tool for their task of managing content and doesn't really know anything about Drupal.

I am Joe Schmoe...or at least I pretended to be, and just this year, I spent a lot of time looking at all the headless CMSes I could find. Why headless? Because practically no "professional developer" today combines the data layer with the rendering layer anymore. With server-side rendering, there is no need, and with projects like Gatsby, Next.js, Nuxt, et. al. it is much easier to grab data from an API, Drupal...well Drupal through a proxy server, render the initial markup, and also have the tight integration with the client-side code.

Furthermore, you work with frontend devs who don't want to work with Twig. You combine data from more sources than Drupal. You do the crazy frontend churn dance that everyone else is doing these days. Doesn't matter if it makes sense, it's where the market went. It's where you have to go if you want a job in 2020. I spent 6 months at the end of 2019 looking for general dev jobs, and I saw way more JS + backend language positions than a strictly PHP or strictly Drupal stack.

I basically skipped Drupal 8 mainly due to where I was working, but I'm not too sad with all the pain I saw of getting Composer in a usable state, early migrations, and even people on 8.x getting left behind without security updates because they couldn't upgrade to the supported releases. Now, I'm working with D9 and it does feel a lot smoother. Using Lando for local dev has been alright. Using the code gen tools to create modules has been nice and in line with other projects. I've been able to handle my CMS requirements, and then some, with only config exports and UI clicks. Once again, very nice...and I know I could add a bunch of contrib modules if my boss needs to suddenly go in a different direction.

What about the competition? Drupal is dead, right? So people must be moving somewhere...Let's give [insert open source headless CMS project name] a spin. Oh wow, that setup was nice. First requirement..okay, easy to implement. Nice. 10th requirement...oh shit, this is terrible. Guess I'll write custom code for a whole subsystem the project hasn't figured out yet. I have not found any other headless solution, written in any language, that compares to the feature set of Drupal. They all fail pretty soon once you get past really simple project requirements. If someone has another advanced headless CMS to try, then please tell me about it.

Even the experimental module process and some of the initiatives, like Media, Layout Builder, and Workflow, I thought were misguided as services can handle media needs and layouts are hard to do in a headless manner. However, I feel I've been wrong each time and am finding those additions to core very helpful now. They set Drupal apart from the rest.

But what about the people? Contributors and new evaluators are important to appease. For some people, I think it is a lost cause trying to convince them of anything. They are bitter, burned, and have every right to join other projects. But for "ambitious digital experiences" where are they to go? For the paradigms they like, WordPress and Backdrop are fine, and I wish Drupal would acknowledge Backdrop more and work with them to appease the disaffected D7 users. I had high hopes for Backdrop and spent a little bit of time in the community, but I was working on too "ambitious" of a codebase I guess for them and felt like my perspective was never going to be taken seriously since it was too in the D8+ realm. If they don't have companies using Backdrop who have budgets to put forth 10% contribution hours, then they have fewer "paid contributors". Drupal has benefited greatly from 10% time and large companies putting in dev time.

So, in the end, I think Drupal should continue the trend of the larger development communities and break down the pieces of the project to make it more maintainable. Keeping everything on d.o is INSANE, especially for simple things like docs. Make a docs site with...gasp...NOT DRUPAL. Use markdown files connected to a repo where you can click "edit page" and make your first contribution to the project...like most every other project I know does. But people keep dumping shit on d.o. and it's confusing as hell for someone new, let alone someone who has worked with Drupal for years.

But I'm not here to do anything constructive other than complain. Hats off to the people doing the dirty work. I know it's tough, especially when people tell you Drupal is dead and the choices you've made have ruined the project. If they can pull off the headless menu component and keep having tight integrations with projects like Gatsby, I still think Drupal will do okay.

Last point...on PHP in general...I've went to many non-PHP meetups, and one issue I tend to have far too often is people literally laugh at you when you tell them you work with PHP and Drupal. They are very pretentious and full of themselves, but they do attend and speak at meetups. I've even heard bootcamp instructors declare "PHP is total garbage" and they know very little about the language...or they point to obscure issues that are never a problem for me or most people. Tell them about improvements coming in PHP 8, and they will rant about how PHP is terrible. Not "oh that's nice since PHP is still widely used on the web and that progress makes the web better". They just metaphorically flip you off. If colleges and bootcamps don't teach PHP and go further to denigrate it, that's another factor contributing to lower numbers on the Update status report.

I see Drupal+contrib as one big thing, since I absolutely depend on multiple contrib modules for the sites I work on. Since Drupal 8, development has become largely by and for agencies and large companies, focused on brochureware and magazine type sites. Every Drupal site I have ever worked on is the opposite of that. It looks like a feedback process is under way, too - the more that what gets done and what doesn't get done is focused on those needs, the more that others leave to find solutions elsewhere, so it gets even more concentrated. Maybe there's a long slow learning curve but it's been already several years.

I've been involved and building sites in Drupal since D4.6, learned PHP through Drupal work, and have both worked for agencies and freelanced. I also knew very little OOP or Composer before Drupal 8. Drupal 8/9 is an absolute dream compared to the previous versions, which I loved at the time. The learning curve (OOP, Composer, Devops, TWIG) was steep, but rewarding and completely worth it. We build small 5 pages sites and large 1000's of page sites in D8. Building systems and leveraging code across projects make it easier. Many custom modifications in D8 can be done with very little code (beyond the scaffolding produced by Drupal Console) compared to D7. Additionally, I don't think the Update Status data is very accurate on D8. My team and I are responsible for 100+ D8 sites that have Update Status disabled since we manage updates through composer in our DevOps system. We don't want clients updating in the UI. I would expect this to be common. If automatic updates within Drupal using composer ever becomes possible we will see a large jump in Update Status numbers for D8 when the module is re-enabled.

There are bumps and hurdles along the way without a doubt but the movement or progress is in the right direction from a code standpoint. My only real complaint is that the Drupal community tends to get distracted too easily by other issues outside of the improvement of the product. These distractions set back the pace of development, expend precious energy, and alienate entire segments of the community. I'm sure anyone who's been around a while can think of a handful of distractions. The most success, equality, and unity will occur from pursuit of a shared goal with enthusiasm. That shared goal is our software.

We don't want clients updating in the UI. I would expect this to be common.

I think this, along with the Composer situation, is somewhere that Drupal needs to make up its mind once and for all to solve a huge amount of complexity/pain for module authors, developers, site builders, and end users.

Currently, we allow installing modules by download+copy, or by Composer. And we allow configuration in the UI, or via config management.

Both of those two points have caused untold numbers of annoying problems and difficult-to-solve bugs and issues that I believe all of us have dealt with. Let's go the extra mile and choose one way or another. If everyone says it's best to use Composer and to only configure things via config management, then we should go that route, and not try to cater to all the wildly different (and often incompatible) use cases. It would make Drupal more consistent in real world usage, and save contributors so much time in having to document how to do almost everything two, three, or four different ways.

Dropping D7 support ensures the slow death of D8+. Helping get the million-ish D7 sites ported with better tools, contrib migration / availability, and (maybe most of all) documentation *might* prevent the decline. D8 had too many incompatibilities all at once, some of them frivolous (like renaming functions).

OOP has made it much harder to understand all of Drupal. There are just too many systems, with too many objects that interact, too little documentation.. That which used to take a few minutes to implement now takes an hour. Or more. Pages that used to load essentially instantly now take tens of seconds. I'm no stranger to OOP, but D8+ is lining up to jump the shark. If it hasn't already.

Please, visit: https://www.drupal.org/CHANGELOG.txt

Main Drupal site is operating with Drupal 7.72, 2020-06-17 !!!

Five years after starting Drupal 8!

At the same time, an extremely important and iconic Drupal module such as Rules is still in version 8.x-3.0-alpha6 released on July 20, 2020. Like many others that were the reason for people to work with Drupal 6 / 7 and gave it incredible flexibility and power - they are in alpha versions today or abandoned.

At the same time, we have Rules that work great with Backdrop CMS - https://backdropcms.org/project/rules.

Like many others, such as Feeds - this module is now in Drupal version 8.x-3.0-alpha9 released on July 12, 2020 and "This project is not covered by Drupal's security advisory policy."

Compare with https://backdropcms.org/project/feeds

Long live Backdrop CMS. Yes, Backdrop's the team may be small, but they are doing what the vast majority of users of Drupal's historic ecosystem need.

There are so many things wrong with drupal. We are all discussing this here on Jeffs blog as there is no real global inclusive platform to make this discussion is starting point, I had so many times questions/ideas I didnt know where to put them.Tthen to drupal.org site which has been "sort of broken" for such a long time and still lacks so many features for effective communication and collaboration. In the end it is "the community" that Drupal is so proud of, there is a community but it is a community that does not SCALE.