It was an absolutely ordinary weekday in the office when I heard my colleague complaining silently that he wish he could use async/await JS language feature in his AWS Lambda code. The bad thing was that at that time (March, 2018) you could only run Node.js 4 and 6 code on Lambda. The good thing was that Node.js 8 support was introduced to Lambda quite soon (it was 2^nd April, 2018) and it looked like the problem faded away. But then Node.js 10 was launched almost immediately (24^th April, 2018) with another set of attractive language and runtime features such as ECMAScript Modules or Worker Threads. How long do we have to wait for Lambda to support Node.js 10 this time? Hold on — looks like there is a way to have Node.js 10 in Lambda already. Let me guide you there.
Isn’t Babel the answer?
Babel is a great project that supports latest JS language tricks such as ECMAScript Modules mentioned earlier. To use it you write some code for Node.js 10 and then let Babel transpile it to something poor current Lambda can understand. And suddenly you realize that its not that easy.
Here is a piece of code that uses async/await:
const { map } = require('async');
const someFunction = async (users) => {const start = new Date();const results = await promisify(map)(users, (async user => {const promisedUser = await Promise.resolve(user);...return something;}));return new Date() - start;}
After transpiling the start variable is put to async call function scope and is not reachable for new Date() — start expression. There are more similar issues with transpiled code — very similar to the one discussed above, some related to async/await alone. I do agree that it is possible to overcome all these issues by sacrificing some syntactic sugar. Personally, I feel bad every time I’m forced to do that.
Third party modules to the rescue
I should be talking about launching Node.js 10 on Lambda now, however, in order to complete the list of possible reasons someone would want to do that I have to briefly mention third party modules. What I mean is, HTTP/2 is one of the Node.js 10 top features, however, third party HTTP/2 module has been there for 5 years. And there are third party modules that could cover Worker Threads or promisify your callback functions. On the other hand, choosing one of these modules now might not be a good decision in a long run. Finally, a need to use latest V8 or libuv version is only possible with latest Node.js version.
Not that childish Node.js child process
So let’s finally get to the point which is based on my previous attempts to try to spawn a child process in Lambda. For example, I was able to start Headless Chromium in a child process in Lambda. Of course, the problem with such approach is that Chromium binary is more than 50 MB in size (does not fit to Lambda deployment package), so as soon as Lambda Function starts it has to download the binary from S3, and that means the whole thing becomes slow. Not to mention that Chromium process consumes hundreds of MBs of Memory. The good thing is that Node.js is not Chromium and it looks like it might be different enough to be run efficiently in a child process.
Node.js is small. Its compressed size is less than 10 MB. In Lambda terms there is more than 40 MB of space left for you code (Lambda package limit is 50 MB).
Node.js is quick to start. I could not find any well written source to support this statement, but it should be obvious from the small number of core modules and widely used third-party modules such as [nodemon](https://www.npmjs.com/package/nodemon). In Lambda terms: my JS code is executed as soon as new process starts.
Node.js only consumes as much Memory as it is required to run application. If application is small — Memory consumption is small. I ran node command (that opens the Node.js shell) on my Linux machine and it consumed 7.5 MB. In Lambda terms: 7.5 MB is not that much compared to minimum amount your Function could be allocated, which is 128 MB of Memory.
Now let’s fit peaces together:
- Node.js 10 is bundled together with your JS code into a Lambda deployment package.
- Upon initial Lambda Function execution a child process is spawned that starts the bundled Node.js 10 runtime.
- Node.js 10 child process executes your JS code.
I am sure that quite a few Lambda fans thought of this idea already, however, I could not find it in any form on the web (NPM to be exact). Therefore, I am proud to introduce you to an open-source NPM module called lambda-node-runtime .
lambda-node-runtime_Run any Node version on AWS Lambda_www.npmjs.com
The schema below summarises the architecture of lambda-node-runtime . It acts as a proxy to deliver handler function call to your Node.js 10 code using IPC and all the I/O traffic through stdin, stdout and stderr channels (Lambda runs your code in Linux environment). Because of transparency of the proxy, your code feels as it is run directly by Lambda.
lambda-node-runtime spawns a Node.js 10 child process and proxies handler function invocation (via IPC) and all I/O transparently
Feel free to find all the specific details in README.md or in source code (really not that much of it) to understand suitability for you Lambda Functions. There are some things you need to be aware of so Benchmarks section talks about some implications of using the module in your Lambda Function.
Is there a future for it?
Node.js 10 will eventually come to Lambda and so please all of those waiting (introduction of latest LTS Node.js version were carried out on April two years in a row, so there is a good chance to see Node.js 10 in Lambda on April next year). The paradox is that JS language evolves fast and so Node.js keeps releasing new versions. It appears Lambda fans will have to queue at Lambda “door” for the latest Node.js release again and again. Until things settle down. While waiting, let’s see if Lambda users will find lambda-node-runtime module useful.