How to use Redis Pub/Sub in Messengers

Written by vgukasov | Published 2022/04/11
Tech Story Tags: redis | pubsub | publish | subscribe | redis-pubsub | messenger | redis-pubsub-in-messengers | coding

TLDRA social network needs to instantly show a new message from another user without reloading a page. The most widespread architecture is: a web browser connects to a server via a WebSocket (a particular type of connection that is established once and is maintained after that) A WebSocket microservice subscribes to the user's private messaging channel. At some point, a web server publishes a message to a channel, and a WebSockets microservice sends it to the browser. Redis implements the Pub/Sub model using the example of private messages in the messenger example.via the TL;DR App

Nowadays, there are a lot of real-time events on most web pages. For example, a social network needs to instantly show a new message from another user without reloading a page. How are systems like that technically arranged?

The most widespread architecture is:

  • a web browser connects to a server via a WebSocket (a particular type of connection that is established once and is maintained after that. Within this connection, the client communicates with the server independently)
  • when an event occurs, a server sends data to a browser via a WebSocket
  • a browser receives the data and displays the event in real-time

The most interesting part of this architecture is server-side. Usually, some microservice handles WebSocket connections. When an event occurs on the server-side, a server sends it to a WebSocket microservice to pass to a web browser.

How to send data from a server to a WebSocket microservice? The simplest solution is to make an HTTP request. This will be enough for small applications with a poor count of events. However, this solution won’t work for most projects because HTTP establishes a TCP connection on each request and sends unnecessary information in headers. For instance, everything will work unacceptably slowly using HTTP in instant messengers where millions of messages are handled per minute.

In this case, a good solution would be to use a message broker. It can be configured for different message delivery models. The Pub/Sub (publisher-subscriber) model is preferable in the messenger example. The WebSocket microservice subscribes to the user's private messaging channel. At some point, a web server publishes a message to a channel, and a WebSocket microservice sends it to the browser.

Pros of the approach are:

  • a subscriber maintains a permanent connection with the message broker
  • a publisher sends only useful information without unnecessary extra headers
  • throughput of the interaction is suitable for high-load projects

Redis Pub/Sub

Let's see how Redis implements the Pub/Sub model using the example of private messages in the messenger.

A websocket microservice subscribes to the messenger events using the subscribe channel [channel ...] command:

127.0.0.1:6379> subscribe messages
Reading messages... (press Ctrl-C to quit)
1) subscribe
2) "messages"
3) (integer) 1

Execution of the command returns 1 if the subscription to the channel was successful and the connection is blocked, waiting for messages.

When a new message is sent, a web server publishes an event to Redis with the publish channel message command:

127.0.0.1:6379> publish messages '{"user_id":123456,"message_id":"f112aa8a-3966-4070-a990-cf7afcdf0eea","message_text":"Oh, hi Mark!","sent_at":1649294564}'
(integer) 1

The publish command returns the number of clients that received the broadcast message.

A message instantly arrives from the WebSocket microservice side:

1) "message"
2) "messages"
3) "{\"user_id\":123456,\"message_id\":\"f112aa8a-3966-4070-a990-cf7afcdf0eea\",\"message_text\":\"Oh, hi Mark!\",\" sent_at\":1649294564}"

1 means action; 2 — channel from where the message has come; 3 message body.

When to use Pub/Sub

Important note: use Redis Pub/Sub only in cases when your server can lose events

If there are no subscribers at the time of publishing, or if the user does not have an open WebSocket connection, the event will be gone. For example, use Pub/Sub for notifications about new messages in a messenger. If a user is online, he will see any change in real-time. But when the user reloads a page, he will receive all data from a server via API.

However, this architecture is clearly cannot be used to send emails or SMS notifications.

Conclusion

  • Pub/Sub model is used in real-time applications
  • Redis effectively implements the Pub/Sub model allowing throughput of up to 1 million messages per second
  • a subscriber subscribes to events with the subscribe command
  • a publisher publishes events with the publish command


Written by vgukasov | Senior SWE at Akma Trading
Published by HackerNoon on 2022/04/11