Building Webiny — A Serverless CMS
Challenges and Lessons Learned Over the Last 12 Months
First, we had metal, then it all went virtual and, finally, we moved to containers. Now, a new era of “servers” is emerging (if we can even call them that); one that is born in the cloud and designed for the cloud.
Evolution of serverless — Jason McGee
Werner Vogels, CTO of Amazon, said:
“Serverless is the future of development”
— and I couldn’t agree more.
However, not everything is peachy. The serverless environment is young(ish), and not all systems we use today are optimized for it, especially the content management systems. Let me explain.
Revolution over evolution
We are still chasing faster horses rather than thinking about how to teleport.
If the previous sentence confused you, let me give you an analogy.
Upgrading your Wordpress version 4 to version 5 is evolution; you have just been given a faster horse. This horse can jump higher, go for longer, and can even walk backward — but it’s still a horse. It’s still the same stack that was invented 15 years ago (initial release: May 27, 2003).
In the meantime, bright new minds have invented the serverless technology, new API standards (like GraphQL), cool and new ways to build and construct the UI with libraries (like React). Today, we even have a serverless database. The non-existence of all these technologies was how the previous CMSs were designed, and those limitations are still baked into them.
The path of revolution requires us to build new things with technologies (and mindsets) that are more powerful, faster, safer, and much more scalable. This is a story about one of those trips.
Why build a serverless CMS?
The answer is simple — I had an itch :)
Joking aside, if you are looking for a CMS built with technologies like React, Node, and GraphQL, and is also designed for the serverless environment — well, you’re in luck!
As of now, there is only one option that’s available today — Webiny.
<a href="https://medium.com/media/931d737448d15cf9a035b99b7008508d/href">https://medium.com/media/931d737448d15cf9a035b99b7008508d/href</a>
Webiny is a CMS for the serverless era. It’s open-source, licensed under MIT, and it also features a hosted version, where you can get your very own serverless environment in just a few mouse clicks.
Webiny has numerous cool features and is aimed primarily at developers. You can learn more about that on the aforementioned website. Now, let’s get back on track with the original title of this post.
Challenges along the way
The challenges we faced can be categorized into three different groups:
- CMS related
- Architecture related
- Business/finance related
We will discuss each category in more detail and then share some useful tips — especially about the business/financial side. Now, let’s dive deeper.
CMS challenges
A CMS is a rather complex set of features that need to work together. Here are some examples.
Page builder
You can’t just create a form with an input field for the title and a CKEditor for the page content and call it CMS. That’s not nearly enough. You need a full-featured page builder — and we’ve built one!
But boy, the effort required to create one that actually works and is optimized is sure hard to describe.
Webiny Page Builder
The first main challenge was managing the data inside the page builder. The interface is complex — each page item has over 20 different properties that you can adjust. In a single page you can easily have over 100 elements, bringing the total amount of props that you need to manage, store, and retrieve up to thousands.
We used Redux to control all that, and it’s not just Redux as a data-layer that saves you loads of trouble, but the debugger that comes with it as well. This wouldn’t have been possible without it. You can move up or down the stack of your events, see how the whole data structure looks in real time, and many other things without which this would’ve been way harder.
Redux flow
Rendering performance
Our CMS has over 100 different moving parts — some hold state, some do major state updates every time you make a change.
Propagating that through the UI can sometimes cause the interface to lag and just not feel “fast” enough. In the beginning, everything was nice and smooth.
You develop one element, you test it, you move on to the next. Everything seems to be working until you finally begin building real pages with tons of elements. Slowly but surely, everything becomes slower and slower.
An obvious first thought is the React Developer Tools. It has the “Highlight Updates” option, which will help you identify the re-rendered elements.
Unfortunately, this will only help you with a very simple UI hierarchy. In our case, this barely scratched the surface.
So, we had to turn to the Chrome Dev Tools, the Performance Snapshots in particular. This tool is priceless when debugging rendering performance in your React app. The recorded snapshots let you see exactly how much time was spent to render each particular element in your app hierarchy (don’t forget to open the Timings section). You can, then, go directly to the specific component’s code and start optimizing things:
Chrome Dev Tools — Performance Snapshot
This is still a very manual process since you have to think about the output and why certain components get re-rendered, but you can quickly find different problems in your app and in some cases solving one problem can fix rendering performance in many places (shouldComponentUpdate and Redux’s connect will be your best friends to get the job done).
Very daunting at the beginning, this process will make you happy in the end.
CSS libraries
Here is a helpful tip which is not very obvious straight away:
If you are using a library for CSS, like Emotion, make sure you do not update props too often on components that control the CSS, as that will create and insert a new style DOM element on each update.
A typical example is a tool to resize elements: the size of the element must under no circumstances be an Emotion prop — it has to be a simple React element style value. Otherwise, you will bombard your DOM with new style elements while resizing is taking place (that’s just how Emotion works).
User interface
Building a UI is a story of challenges enough to fill a whole book, if not several ones. Luckily, when we started, Google had just launched Material Design, and there were a few react libraries popping up that had those components ready to be used. The one we used is called RMWC, and, together with the Design section on the material.io website that teaches you how to use their components in the right way, we managed to create a pretty decent UI for our CMS.
Themes and plugins
What is a CMS if you can’t build your own theme, or customize it with your own plugins and add-ons?
We wanted our theme system to be super simple. Since the page builder is actually the place where you build the content and generate the HTML, we managed to create a theme library where all you have is a small JSON config and everything else is done through (S)CSS. You can learn more about our theme setup on our documentation website.
{
layouts: [
// defines a list of layouts
],
fonts: {
// defines font faces
},
colors: {
// defines a list of default colors
},
elements: {
// defines element settings
},
typography: {
// defines typography styles
}
}
As for the plugin system, it’s hard to find any good advice or best practices on how to make a plugin system. So, what we actually did was we made the whole CMS as a set of plugins. Every button, menu item, and form element is actually a plugin.
This made us feel comfortable in the scalability and the possibilities of what you can do with plugins. On the other hand, this made our code very modular and decoupled. Finally, this has a great effect on developer experience (DX) since there is no difference in how the code is written for the “core” of the system versus how the code will be written by other developers when building their own plugins.
Architectural challenges
Architecture for a serverless application is rather simple, right? You have an API Gateway, some Lambdas, a database on the other end, an S3 bucket with static files.
But, here’s a question for you — what about a multi-tenant serverless application? Basically where each user can have its own domain name, its own SSL certificate, static file hosting, and a database.
The hosted version of Webiny is exactly that.
And, I can tell you that, today, there isn’t a cloud provider that will allow you to have 10.000 different SSL certificates on an API gateway or serve static assets from the same bucket using multiple domain names. CloudFront and similar CDNs have hard limits on the number of SSL certificates and domain names that they can serve. We talked to several cloud providers and they just don’t support that.
So, although serverless is great — it’s got its limitations. Or does it?! ;)
We took this problem as a challenge and, to get around it, we used an OpenResty Nginx fork and wrote our own API gateway for the AWS Lambda, designed our own SSL certificate management system in the front, created a proxy to the S3 buckets, and also added a bunch of usage and performance monitoring agents, et voila — we can now support as many serverless tenants as we want.
OpenResty is great; it just takes a bit of practice with coding in Lua.
https://github.com/openresty/lua-nginx-module
You might argue that this is not a true Serverless CMS since it uses a “server” in the form of a proxy to make it work.
Technically, the proxy is not required; it’s only needed for the hosted environment with multiple tenants. But, also, for the sake of argument, if you use a CDN in front of your serverless page, like Cloudflare, those are technically OpenResty reverse proxies.
Don’t get me wrong, this definitely is an area where big cloud providers like AWS, Google, and Azure need to improve since those platforms, today, don’t support a multi-tenant serverless architecture.
Business challenges
By this point, you’ve hopefully gotten the picture of the effort required to build a serverless CMS, but the story doesn’t really end there — I’ve got one more thing to share.
Webiny is open source, but it’s taken a lot of time, dedication, and money to create Webiny. When we started the project, we knew doing this on the side was never an option; we would most likely fail in our attempt. So, the only way to build something as large and complex as this required financial support. But, no investor will give you money if you go and approach him with an elevator pitch to build a CMS, which has 0 users and no revenue — it would just never be convincing enough to raise investment.
Also, creating a CMS 10 years ago was way easier because the bar to enter that market was much lower, but that’s no longer the case. Several people told us — “Hey, just build an MVP.”
Well, you go and try to build an MVP for, let’s say, an electric car, and let me know which things you will leave out that your competition already has today.
Let me help you with answering that: you won’t leave out a single thing! Your car will have everything the competition has and then some — and that’s when you’ll launch! Otherwise, nobody will buy it.
No matter what you do in life, if you want to be great at something, you need to reach the level of the “current greatness” and then be slightly better than that.
So, how did we solve our financial problem?
Well, the team originally started as a small web agency. We earned some money by doing standard web-agency stuff and then, at one point, we decided to stop accepting new work, all the money left on the bank account was used as an investment to ensure salaries for the team so that we can be fully committed to this project. Had it been any other way, Webiny would never have seen the light of day.
One small tip in this area.
AWS has the Activate program, you can apply it and get $1000 in AWS credits, which is great for covering the cost of the infrastructure while developing.
Another thing to explore is things like ProductHunt Ship. If you subscribe for the annual membership ($600 — $1500), you can get between $5000 and $7500, also in AWS credits, alongside other benefits. Note that if you apply for the AWS activate, the initial credit amount might be deducted from the credit amount you get from ProductHunt Ship.
What’s next
If you got this far … I admire your focus. I blame the length of this post on my long plane flight.
Anyhow, hope you’ve enjoyed the read. I would really appreciate it if you’d give Webiny a spin, and let me know what you think of it. You can reach out to me via twitter @SvenAlHamad.