AWS recently introduced SQS integration to Lambda. We have been waiting for this for a long time now. In the following post, I’m going to share a quick-and-simple tutorial on how to get started with message distribution between Lambda functions, using SQS.
In addition, I will also compare between SQS and SNS (i.e. why should we choose one over the other), and present an in-depth performance analysis of using SQS as a message distributor.
Setup
In this tutorial, I will use the Serverless Framework, to send a message from one Lambda function to another Lambda function via SQS.
Prerequisites:
- npm installed.
- Serverless Framework installed.
- Python — Feel free to use your favorite programming language.
- AWS account (duh).
Let’s start by creating the serverless project:
serverless create --template aws-python3 --path sqs-lambda-tutorialcd sqs-lambda-tutorial
At this point, we can configure our serverless.yml and handler.py files. The code can be found in the following GitHub repo.
serverless.yml
This yml file configures our functions, permissions and SQS resource.
handler.py
This is where our code runs. It contains two functions. Each function represents a different Lambda function. One will be triggered manually, and the other one will be triggered by an SQS message.
Deploy!
This is the magic of Lambda. Just run the following command while you are in the same folder as serverless.yml:
sls deploy
The output should look roughly like this:
Now we can invoke the “start” function by running the following commands:
sls invoke -f start-lambdasls logs -f start-lambdasls logs -f end-lambda
You will be able to see the CloudWatch Logs of the Lambda functions with the SQS message.
We are done. Congrats! 🎉👏⚡️
Event structure
If you plan to use SQS in your application, you will need the event structure and data. You can find it in the following Gist:
SQS vs. SNS
When applied to Lambda only, this is a tough call. Up to this moment, only SNS supported triggering Lambda functions. Now both SNS and SQS can trigger. Let’s compare the relevant parameters for Serverless applications:
The main advantage for SQS is the batch messaging, which can also lead to cost reduction.
Performance analysis
Distributing messages between services can be complicated. Understanding the performance impact on our applications is almost impossible. At Epsagon, we love to see the bigger picture for highly distributed applications. In a previous post, we compared several methods of distributing messages between functions (direct sync invocation, direct async invocation, SNS and Kinesis), and now, using our serverless performance monitoring tool, it’s a great time to introduce the results for SQS:
- Operation duration (how long we wait to publish message): ~10ms.
- Total duration (how long it takes from publish until it reaches the destination): ~45ms.
Compared to the results from the previous post — this is much better!
Conclusion
Lambda integrations are great. The more, the better. In this post, we learned how to deploy Lambda functions that distribute messages using SQS.
We also saw an updated comparison between SQS and SNS for Serverless applications and understood the performance implications of distributing messages between functions.
Considering features, cost and performance, SQS is the best choice for the task of message distribution between Lambda functions.
Originally published at epsagon.com.