Bigpipe

In a recent blog post by Dries Buytaert, the founder and lead developer of Drupal, he discussed various aspects of the "decoupling movement", pros and cons of various approaches, and how it relates to Drupal and its future design architecture. In his own words, "the term decoupled refers to a separation between the back end and one or more front ends instead of the traditional notion of service-oriented architecture."

It's an interesting topic that has captured a great deal of developer attention over the last several years, primarily because of the remarkable shift in how people look at the underlying structure of websites and how that relates to what content consumers expect from the Internet. So, if you have a chance, I recommend you read that blog by Dries and get a sense for where he sees Drupal fitting into that paradigm. 

But what I wanted to write about was a system and concept that Dries mentioned in the middle of his blog, that is the performance ramifications of this type of paradigm, and more specifically systems like BigPipe that are being incorporated into Drupal 8 to boost rendering efficiency dramatically.

So, What is BigPipe?

BigPipe is a browser rendering system engineered by Facebook, which was instrumental in remarkably boosting the page rendering efficiency of Facebook.com

We call the whole system BigPipe and it allows us to break our web pages up in to logical blocks of content, called Pagelets, and pipeline the generation and render of these Pagelets. Looking at the home page, for example, think of the newsfeed as one Pagelet, the Suggestions box another, and the advertisement yet another. BigPipe not only reduces the TTL of our pages but also makes them seem even faster to users since seeing partial content earlier feels faster than seeing complete content a little bit later.

TTL stands for "time to load", and is the key metric driving that work and some of the issues Dries addresses in his blog post. Time to load is critical for website usability and user satisfaction, since you typically only have mere seconds to capture the interest of visitors before they leave. If your website takes 5 to 6 seconds to load, you've likely lost them for good. So, clearly any kind of change in architecture or design patterns that reduces TTL to the smallest amount of time possible is likely a good thing.

Drupal Already Has Caching Systems, How Does This Help?

During Drupal 8's long development there have been lingering concerns about performance. The codebase for Drupal is growing tremendously with version 8, in terms of total files and also lines of code, so it stands to reason that the likelihood for increased processing time is there. Of couse once you enable the built-in caching systems and the typical stable of optimization tools like opcode caches and proxy caching on the server, you can improve the situation a lot. There are even some great contributed modules which can significantly enhance Drupal Core caching.

But highly individualized content will always remain a more "expensive" type of content to render since much of it is specific to a unique viewer, and as such caching has limited returns in performance savings. Additionally, these more expensive pieces of content may potentially hold up the rendering of other less-expensive (cached) content components, maybe even delaying their rendering even though it could have been done in a fraction of a second.

Drupal 8 will be integrating BigPipe in order to solve this problem, and according to Dries "Drupal 8 is the only CMS with BigPipe deeply integrated across the board for both core and contributed modules".

With BigPipe, an approach for client-side dynamic content substitution, we can render our pages progressively, where the skeleton of the page loads first, then expensive components such as "songs I listened to most in the last week" or "currently playing" are sent to the browser later and fill out placeholders. This component-driven approach gives us the best of both worlds: non-blocking user interfaces with a brisk time to first interaction and rapid piecemeal loading of complete Drupal pages that leverage the theme layer.

To see what this could look like in terms of performance, check out this short video demonstrating page loads for Drupal 8, with and without BigPipe and with cold and warm caches.

The Future for Drupal 8 is Bright, or At Least Much Faster

As you can see from that video, incorporating the BigPipe system into Drupal 8 page rendering results in a remarkably faster time to load. That kind of responsiveness is critical for maintaining higher levels of user engagement, and not only helps keep visitors from drifting off the website, but also increases the feel of the website being more like an app, rather than a distant and slow content store.

Will this kind of system require more work from developers to make it work? According to Dries, module developers will need to provide some "cacheability metadata", in the form of properties like tags, context, and age, but Drupal 8's Dynamic Page Cache module should do the bulk of the work. So, If the improved rendering system works as well as it looks and the caching metadata as easy to implement as documented, then Drupal 8 looks to have a great chance to really shine as a CMS platform.