Making Domino “Real Time”

In my last post I spoke about some technologies for real time operations. In this post I would like to take it a bit farther and talk about making Domino a “Real Time Database“. I use that term rather lightly as there is no means of scheduling and prioritizing transactions thus Domino isn’t really real time, but only behaves as real time.

The Pieces

First off we developed an Event Bridge plugin that taps into Domino’s C API for events which are fired when documents are created, updated and/or deleted. This is the most important piece of this puzzle. It provides a hook into the Domino event model to drive all sorts of functionality. Things like updating search indices, notifying pertinent workflows, etc.

If you recall in the previous post we spoke about the Redis PubSub system. This was another plugin we developed to inform the Redis PubSub engine of these events. We also spoke about a WebSocket server and client connected to the WebSocket server. These are also required for this technique to function.

The Pattern

So our plugin publishes events when a document is created, updated or deleted then notifies the Redis PubSub system. Redis then passes that information down to any clients which may be viewing the affected document. This allows us to automatically update said document IF the document is being read somewhere else. If it’s being edited (or is dirty), we can notify the user that a change has happened and they can either accept or deny the change. A third option would be to keep track of what a user has changed and if the user hasn’t changed what was changed elsewhere, then just update it.

The Scenario

Consider the following scenario:

We have a user (user 1) that opens a document via a browser or some client other than Notes. We have another user (user 2) that opens the same document in the notes client. User 2 makes a change to the document and saves it. User 1’s view of the document just updates automatically with the new changes.

Now another user (user 3) opens the same document via a browser or some client other than Notes, puts the document in edit mode (user 1 is notified) and then changes a field value, we can then notify the participants of the document (clients other than Notes, via Redis PubSub) and just update their instance of the document. However this causes a bit of a conundrum, what do you do if User 3 cancels or abandons their changes? You’ve already updated other instances of the document with the new values User 3 defined. You’ll need to notify the other clients that User 3 cancelled their changes and revert back to the original document in the other instances.

Putting it all together

The WebSocket server

The WebSocket server listens for 4 events:

  • open – Fired when someone connects to the WebSocket server
  • message – Fired when a message from a client is received
  • error – Fired when there is a communication error
  • close – Fired when a connection is closed

And has 2 methods:

  • send – Send a message to a connection
  • close – Close a connection

There are 2 events we’re interested in. First is the open event, this starts the handshake process described in the previous post. The next one is message. This is where we can start to hook into logic based on what type of message is received. Determining the message type is where we have to start putting some thought into the structure of the messages that are being sent via WebSockets. Me personally I defined a TypeScript Enum for all the different types of messages along with a TypeScript Interface for each. This ensures our messages have the properties that we need them to have.

export enum NotificationType {
    connect = 'ws_connection',
    user = 'ws_user',
    joinDocument = 'join_document',
    documentFieldUpdate = 'document_field_update',
    documentUpdate = 'document_update'
}

export interface WsNotification {
    type: NotificationType;
    toChannel: string;
    msg?: string;
}

export interface DocUpdate extends WsNotification {
    user: string;
    replicaId: string;
    unid: string;
}

export interface DocFieldUpdate extends DocUpdate {
    fieldName: string;
    fieldValue: any;
}

export interface DocReadersEditors {
    readers: string[];
    editors: string[];
}

export interface JoinDocument extends WsNotification {
    from_user: string;
    currentParticipants?: DocReadersEditors | string[];
}

A short explanation of what this is. These are the interfaces defined for each NotificationType enum (I skipped the connect and user for brevity). The interfaces define what properties are required and/or possible. By defining these Types our editor and the TypeScript compiler will enforce these rules ensuring we don’t miss anything.

The WsNotification interface above has a property called toChannel which is a required property. This would be the name of a user or a Redis PubSub channel and informs the API of where to send a notification. Also notice the type property and that every other notification type extends this interface.

Now for a WebSocket server, we can check the type property of a WebSocket message and do something based on that type:

onMessage(message: IMessage) {
    if (message && message.utf8Data) {
        let msg: WsNotification = JSON.parse(message.utf8Data);
        let type: NotificationType = msg.type;
        switch(type) {
            case NotificationType.documentUpdate:
                //Do Something
                break;
            ...
        }
    }
}

This is pretty straight forward I believe. In the //Do Something block of the switch statement above, I normally fire a node event here and have a handler that does something, be it persist info to Redis or Domino (via REST) or just pass along the notification somewhere else. But I try not to include any more logic here than absolutely necessary. What code is actually run should be defined elsewhere.

The WebSocket Client

Fortunately the WebSocket client follows pretty much the exact same pattern and events as the WebSocket server. While our client may or may not be able to execute node.js functionality we can still use an event model of some sort to ensure things happen. Also, we can use the Types we defined for the server in the client code. As a matter of fact, it’s much better if we are able to share those Types as that’s one less place to make a mistake. I would also say that the ability to share those Types will set one of the key requirements for the front-end framework, that it must be module based.

Wrapping Up

Hopefully I was able to show that to use this technology really isn’t that big of a leap. To me the hardest part was coming up with a pattern of operations and what has to happen to make sense out of the passing around of all these WebSocket Notifications. Honestly, the hardest part of this is defining the rules around each Notification type and what happens on the client when one of these Notifications are received. The actual usage of WebSockets is quite easy to understand and implement, it’s the logic around everything else that’s the difficult part.

Until next time… Happy Coding.

Share This:

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.