How Libuv Thread Pool Can Boost Your Node JS Performance

In this 5th instalment of my “Node JS Performance Optimizations” series, I show you how to increase Node JS performance with thread pool management. We achieve this by understanding how Libuv works, how the thread pool works and how to configure the number of threads based on your machine specs.

Are you a Node developer who’s not yet familiar with the inner workings of Node JS? If so, you might be deploying production applications with some default configurations that existed since installing Node. In this article, I’m going to touch on a lesser-known configuration that could very well double the performance of some of your application’s operations. This will depend on a number of factors, but chances are good this is going to be a win for many.

The Node runtime environment is made up of a few moving parts. We should all be familiar with the Google V8 engine, which is responsible for executing our JavaScript logic. There is, however, a lesser known library called Libuv, which is responsible for managing asynchronous I/O operations.

These I/O operations are also well known as “heavy duty tasks” that related to the Operating System. Tasks such as files and folder management, TCP/UDP transactions, compression, encryption, etc. are handled via Libuv.

Now, while most of these operations are asynchronous by design, there are a few that are synchronous, and if not handled correctly, could cause our applications to be blocked. It is for this reason that Libuv has what is called a “Thread Pool”.

Libuv initiates a thread pool of 4 threads that it uses to offload synchronous operations to. In doing this, Libuv ensures that our application does not get blocked unnecessarily by synchronous tasks.

It is here that we will take advantage of a setting to better suit the specs of our machine or the virtual machine that our app will be deployed to. This is because we are allowed to change the default value of 4 threads to anything up to 1024 threads. We achieve this by setting the UV_THREADPOOL_SIZE Node variable.

Physical vs Logical CPU Cores

To better understand what to set the UV_THREADPOOL_SIZE to, we need to first understand how many logical cores our machine is running. If we take my MacBook Pro as an example, it’s running 6 Physical CPU cores (Intel).

However, these cores have hyperthreading, which means that each core can run 2 operations simultaneously. We therefore regard 1 physical core with hyperthreading as 2 logical cores. In my case, my MacBook Pro is running 12 logical cores.

The recommendation is to set the UV_THREADPOOL_SIZE to the number of logical cores your machine is running. In my case I will set the thread pool size to 12.

It makes no sense setting the size to anything more than the logical cores your hardware is running and could actually result in poorer performance.

When it comes to deployment, the last thing you want to do is manually set the UV_THREADPOOL_SIZE, as your app might run on multiple environments with different machine specifications. As such, we need a way to dynamically set the thread pool size the moment the app is launched on the relevant environment.

The good news is that this is quite simple, but must be dealt with caution. To achieve this, add the below code to the top of your root JS file of your Node application:

const OS = require('os')
process.env.UV_THREADPOOL_SIZE = OS.cpus().length

The OS module is native to Node JS. It has a function cpus(), which returns the total amount of logical cores your machine is running. What’s nice is, should your CPU cores not have hyperthreading, this function will just return the number of physical cpu cores instead, which is perfect.

I trust this article proved valuable. I recommend watching the embedded video as well as checking out my Source Code repo on GitHub which has code samples of everything mentioned here.

Till next time, cheers 😎