This is a story of how the load time of the UrbanClap website went from 13+ seconds to less than 5 seconds (3G Singapore Server on Mobile) in a month. And to corroborate that, let me start with some screenshots.
This is where we were before we started this performance exercise:
And below is the progress we made.
Webpage Test(urbanclap.com/delhi-ncr-wedding-photographers), Singapore — EC2 — Chrome — Emulated Nexus 5–3GFast — Mobile (Jan 2017)
Background
We are a (mostly) responsive webapp using ReactJs with server side rendering, and webpack (v1) as our bundling tool. We also have a varnish caching layer on top to reduce our server load, and to further reduce our Time to First Byte(TTFB). For details on our tech stack and choices, you can read this post.
Start of 2017, we took on a month long project to improve our website performance, specifically the first time load of our listing pages (eg. www.urbanclap.com/delhi-ncr-wedding-photographers). Why did we choose these pages? These are our Google Search listing pages, which means most of our website traffic enters through here. Why specifically the first time load? Most of the users are unique and come through google search. So paying special attention to the first time load is extremely crucial when the user visits without any expectations. First impression could indeed be a last impression. Also, its well documented that the Google Search Engine prefers sites that are better performing and load in lesser time.
We researched a lot of resources, tried a lot of ideas, and did a lot of things. With the mammoth task now over, I listed down all the things we did (with helpful resources) for the benefit of rest of the community.
One important thing to note is that we just can’t be blinded by website performance. We still need to make sure we don’t hamper product, user experience, analytics etc. I can’t stress this enough. It’s easy to go overboard with engineering, but at the end of the day you are solving a business problem with end user and business in mind.
With that in mind, I have tried to bucket all the things which we did in roughly two areas: ship less assets, change order of things. In part 1 of this two-part series, I will talk about shipping less.
Shipping less assets
You can be as clever as you want about how you order the resources you send, but at some point, the bottleneck will be the amount of resources you are sending. Every single line of js code you write, external script you include, css tag you add, image you render on the page has to be not only downloaded, but also processed by the browser. This becomes especially important on mobiles (roughly half of the traffic comes from mobile, so don’t ) — connections are often shoddy (3g connection is a good scenario; there is still 2g and edge), hardware could be cheap and phone takes time to process css and js — all of which cannot be ignored.
Stats for urbanclap.com/delhi-ncr-wedding-photographers
I will try to list down all the things which we did to trim down what we were shipping to the browser. The overall result of these steps is summarized nicely in the adjoining image.
1. JS Chunking
If you are using react-router, it is relatively easy to create route specific chunks via webpack's code-splitting. Here is a pretty good tutorial on how to achieve that with require.ensure. Make sure to name your chunks. When you have a lot of routes, it makes things a lot easier.
// creates a code split, and then asynchronously gets the js file// for that route.<Route name="details" path="/details" getComponent={(location, cb) => {require.ensure([], (require) => {cb(null, require('./containers/Details/DetailsPage'));}, 'detailsChunk');}}/>
2. CSS chunking
While js chunking comes out of the box with webpack, for css chunking (ie different css for each route) you need to have some workarounds. For small apps, it might not be worth it to do css chunking. But for an app our size, it almost became a necessity. To achieve this, first you have to set multiple entry points in webpack config, one for each route you want to have a separate css for. Each entry point would now produce its own js and css.
// produces detailsChunk.js and detailsChunk.cssentry: {...'detailsChunk' : ['./src/containers/Details/DetailsPage.js'],...}
Finally it requires a bit of trickery to include the css for a route. Not only you have to get the name of the css file for that route, but also insert it as a stylesheet (make sure its browser compatible) in your document.
// requireStyle - gets the name of the css file (detailsChunk.css)// and adds it as a stylesheet to the document as here:// https://github.com/guybedford/require-css
<Route name="details" path="/details" getComponent={(location, cb) => {require.ensure([], (require) => {requireStyle('detailsChunk', () => {cb(null, require('./containers/Details/DetailsPage'));})}, 'detailsChunk');}}/>
Note: An obvious question would be that some common app and vendor libraries (like React) would be included in every chunk. The solution to that is to create a separate chunk for it. CommonsChunkPlugin is your friend here.
3. Mobile vs Desktop
The age old question: Whether to go responsive or adaptive. We can have a longer debate on it but whichever way you go, you need to have a UX which is specially designed for mobile. So whether you handle it with tons of media queries as in responsive (which means more unneeded code being shipped for a certain device type), or keep js and css separate for mobile and desktop (which means possibly more dev resources needed), its upto you. But since our page had significantly different design (even different cover images for mobile and desktop, which meant we had to download both images for both platforms), we decided to finally completely separate out our mobile and desktop components and bundle them separately. This helped us set up an infrastructure where by default our pages were responsive, but we could change any route to serve device specific components if need to.
First we had to split our code itself into desktop and mobile. A simple following file structure helped achieve this, enabling room for code refactoring between desktop/mobile components.
containers--- Details---DetailsPageDesktop.js---DetailsPageDesktop.scss---DetailsPageMobile.js---DetailsPageMobile.scss---DetailsPageCommon.js
**DetailsPageDesktop.js**
import {} from 'DetailsPageCommon.js';var styles = require('DetailsPageDesktop.scss');
class DetailsPageDesktop extends Component {...}
Then we had to create separate chunks (js and css) for mobile and desktop versions of the route. We did this by tweaking our entry points in webpack config.
// this will create detailsChunkDesktop.js, detailsChunkDesktop.css,// detailsChunkMobile.js, detailsChunkMobile.css// detailsPageCommon will be automatically included in both the js// chunks.
entry: {...'detailsChunkDesktop' : ['./src/containers/Details/DetailsPageDesktop.js'],'detailsChunkMobile' : ['./src/containers/Details/DetailsPageMobile.js'],...}
Finally, we have to tweak our routes config to resolve to different components for mobile and desktop.
// isMobile() -> Make sure this function works on both client and// server side. Easiest way is a regex on user agent.
<Route name="details" path="/details" getComponent={(location, cb) => {if (isMobile()) {require.ensure([], (require) => {requireStyle('detailsChunkMobile', () => {cb(null, require('DetailsPageMobile.js'));})}, 'detailsChunkMobile');} else {require.ensure([], (require) => {requireStyle('detailsChunkDesktop, () => {cb(null, require('DetailsPageDesktop.js'));})}, 'detailsChunkDesktop');}}}/>
We faced an additional problem: we were caching our markup on varnish based on url. Since desktop and mobile devices would have different markup, we had to make varnish cache device aware.
4. On demand chunks
Sometimes even for the same route, we might have components which are heavy and don’t need to be shipped with the current route. These can be loaded on demand, either after visit on the page or after an event handler (click of a button). Any sort of dialogs/modals (where the route doesn't change) or below-the-fold components are prime candidates for this. For such components, we can create separate chunks (with its own js and css), which only get downloaded when needed.
First, we create separate chunks for such component in the entry in the webpack config as described earlier. Then, we trigger the asynchronous retrieval of these chunks when required.
// splits code for this component, and also asynchronously gets the// component. Returns a callback with the component.
function loadHeavyDialog(cb) {require.ensure(['./components/HeavyDialog/HeavyDialog'], (require) => {requireStyle('heavyDialogChunk', () => {const dial = require('./components/HeavyDialog/HeavyDialog');if (cb) {cb(dial);}});}, 'heavyDialogChunk');}
**DetailsPage{Desktop}.js**
// on a button click to load the dialog asynchronously and ensuring // this component doesn't get shipped with the parent route.function onLoadHeavyDialogButtonClick() {// show loader possiblyloadHeavyDialog((dialogComponent) => {// do something with the component now.// hide loader.});}
For further optimisation, you could get this component to be automatically downloaded after the document has been loaded for the route. This will ensure your user experience isn’t adversely affected; now the user doesn’t have to wait for chunks to be downloaded when the need for them actually arises.
<Route name="details" path="/details" getComponent={(location, cb) => {require.ensure([], (require) => {requireStyle('detailsChunk', () => {cb(null, require('./containers/Details/DetailsPage'));// pre-emptively get a component which we know we might// use later.loadHeavyDialog(null);})}, 'detailsChunk');}}/>
The same strategy could also be used for route based chunks which have a high chance of being visited from the current route.
5. Size/Quality of images
For media heavy pages, cut down the size of images you are shipping. Apart from the usual image compressions, make sure the size of images you are requesting is what you need. Be extra strict on mobile, you will be surprised by how much you can manage with smaller images. We were able to reduce the media content being downloaded on mobiles by more than half without affecting any product metrics.
6. Remove unneeded fonts
We were earlier using 3 custom fonts, 2 for rendering text, and 1 for icons. To prevent FOUC, fonts are pretty much considered as a render blocking resource by the browser. After working with the design team, we cut down to just 1 custom text font and 1 icon font. We went even stricter for mobile and removed the need for the custom text font, using just the system fonts. The form factor is so small in mobiles, that the additional benefit of using custom fonts is negligible. So we were able to render content much faster on mobiles.
7. Be choosy about third party/open source libraries
Third party libraries are great. But they normally come with a lot more functionality than you need. Be picky. Use them for motivation, and then either implement your own or be careful with what you need from them. Examples of some common third party libraries whose usage needs to be looked at are:
- Lodash: Whenever you see import * from lodash, it's a bad sign. The lodash library is a good 10kb (gzipped) in size and usually you only need a few functions and not the whole library. Luckily, lodash provides support from that and is very well documented here.
- Material UI: No doubt material ui is a great place to start, but it is very heavy. We had a few components which were using material ui, but we were able to easily mimic them, saving quite a bit of size in the process.
- babel-polyfill (or babel-runtime): Do you need babel-polyfill? Short answer: no, if you are already using babel-runtime. Removing the dependencies on babel-polyfill can significantly reduce your bundle size. Here is an excellent piece that explains the use of one vs the other.
- moment.js: Once again, a very popular, but a heavy library given the amount of code you actually use out of it. Here is a good discussion on how to regulate its size.
8. Reduce api content/react redux store content
This only applies if you are doing server side rendering. If you are, you should be familiar with the server sticking a replica of its redux (or whatever store you are using) in the bottom of document. If your page relies on a heavy api (ours was), there could be a significant addition to the payload of the initial document. On careful inspection, we realised there was a lot of unneeded data in our store and were able to trim that.
9. Keep analysing your chunks
While all of this is a good one time effort, over a period of time, developers will often become careless (especially in a bigger team) and won’t be disciplined or aware about implications of bundle size. Here are some helpful tools/tips:
- Webpack Analyzer: The Best way to look at your bundle sizes.
- webpagetest.org (or Chrome Dev Tools): Gives a good synopsis on all the assets required by your site, and sizes of different types of assets.
- Periodic Monitoring: We wrote a quick utility to track bundle sizes over releases. Housing has a great post on continuous integration too.
That's all for now. Ofcourse, a lot of these things will add only a small value. But performance is rarely ever achieved via just one thing. It's a combined and a continuous effort of a lot of small thing. If you have other such tips to reduce website code size, do let me know in the comments below!
The next part of this series will discuss how to chain your assets in a way that the most essential assets are downloaded/executed as soon as possible, providing further boost to performance.