Right after Collabsphere 2018 I was introduced to a couple of new technologies. These were WebSockets and Redis. I must say that since discovering these technologies my mind hasn’t settled even a little bit. This has opened up a whole new world of possibilities for DIG and other projects I’ve been working on. So, what exactly do these things do?
This technology has actually been around for a while, but I’ve never had the time or thought I had the need to investigate it. Man was I wrong, this is a technology I should have made the time to learn and implement. Normally when developing a web application we’re dependent on HTTP. Now, HTTP works on a per-request basis. The browser connects to the server, makes a request, the server returns a response and the connection is then closed. With WebSockets this is a persistent connection, meaning once the client connects, it stays connected and can send data to the server and the server can send data to the client as things happen without having to make a request to the server. So imagine if you will that some event happens in domino which prompts the WebSocket server to notify the client of this event. Then the client can take action on it or not.
A perfect example using Domino would be some routine in a plugin running a long process and as that process is running it’s sending progress information back to the client. The client can then update a progress bar or whatever as it’s happening on the server. Pretty cool huh? No polling the server (are you done yet, are you done yet, etc.), no waiting on the next request, no hoop jumping what-so-ever. Just Real Time information that the client needs in order to complete a task or notify the user.
Another example which gets used most often is a chat client. You want everyone in the chat to know who is in that chat. So, when someone joins you notify all the other clients that someone has joined. Then all messages will flow through the WebSocket server down to each client in real time. The key term here being Real Time.
Redis is actually a string key store database. You define a key that has a value. That key and value is stored as a string. It does also have the concept of Lists and Sets, but these values are stored as a string (I can’t specify that enough). So any value stored in Redis needs to be parsed through
JSON.stringify. Redis also offers a PubSub engine, which is extremely scalable. The idea around the PubSub is a user
Subscribes to a PubSub Channel. Then when something
Publishes to that channel, everyone subscribed gets notified. So, with that bit of information let’s go back to that event from Domino. Imagine if you will that event Publishes something to a Redis PubSub channel. The PubSub Channel then notifies all the users that are subscribed to that channel and their client can do something, say reload a document they’re viewing.
Putting it Together
Putting this all together to form a solution consists of the following:
You’ll need the WebSocket server class which manages connections. When someone connects we perform a small handshake which is made up of a request to the client to identify it’s user. The reply to that gets stored in a Map where the user’s connection is the key and the user is the value. Now we can reliably identify a user based on their connection to the WebSocket server. This class is also responsible for firing events based on messages received from clients.
Next is the EventHandler class, this is just a class of event listeners that react to events from the WebSocket server.
Next up is the User class. This class of course contains information about the user, but also stores all of their WebSocket connections (they may connect from multiple clients). It is also responsible for sending messages to the user’s connected clients. This class also contains the Pub and Sub Redis clients with event handlers on what to do when that User subscribes to a channel and when something is published to a channel the user is subscribed to (usually sending a message to the user’s connected clients).
The last bit is the RedisDb class. This class is responsible for persisting any key values to the key store and Domino.
With these 4 classes you now have a Real Time environment for which to send information back and forth. Things like online statuses, members of a chat, chat messages, people viewing a document, populating an activity stream, disabling some selection somewhere because a process is running on it and re-enabling it once the process is done. We can also stream audio and video via the WebSocket connection. The possibilities here are endless.
All of this runs on Node.js via an Express Server with a WebSocket Server module. All of this functionality makes up less than 1000 lines of code. It’s endlessly extensible, performant and scales very nicely (had a couple of clients in an endless loop and nothing skipped a beat on the server side). I’m still very excited about these technologies and starting to push the boundaries of what we can do with Domino. Once v10 is released, I’m sure the boundaries will grow quite a bit. These are exciting times.
Until Next Time, Happy Coding!