TMB: Not so simple message bus

Background

A message bus is a mechanism for sending messages between peers. One or more peers, called publishers, send messages to the data bus. One or more peers, called subscribers, receive messages from the data bus. A peer can be both a publisher and a subscriber.

In previous articles I have shown a very simple message bus, too simple to be of real use. The message bus in this article is more useful.

TMB it is the a stripped-down version of an industrial-strength message bus that I have used in commercial projects. Parts that have been removed include encryption, authentication, authorization, various optimizations, and communication methods other than TCP/IP.

The Text Message Bus

In the Text Message Bus, TMB for short, all peers communicate using TCP/IP. The message bus is implemented by as a TCP server listening for new connections on port 4711. Each peer is a TCP client.

Messages

Each message is a single line of text, ending with a newline character. If the server receives a line that has a carriage-return character before the newline, then the carriage-return is removed. This is the only modification that the server ever does to a message.

Peer messages

Peer messages are the messages sent between peers. They start with a colon followed by a category and whitespace. The category is the only part of a peer message that the server looks at and interprets. After the category and whitespace comes the rest of the message. The server does not look at this part.

Messages are passed on to subscribers unmodified.The exact same bytes that were received are sent on to subscribers, including the leading colon and trailing newline.

Here are some possible messages, taken from the example program:

:/upper/shape square Category is /upper/shape
:/upper/colour red Category is /upper/colour
:/lower/shape circle Category is /lower/shape
:/lower/colour yellow Category is /lower/colour

While to you and me there seems to be an obvious structure to the category (some kind of object, and an attribute of that object), the server does not think so. The structure and interpretation of categories is up to the peers. The server has noy view and no say, except that the category must not contain whitespace.

So what is the category for? It is used for deciding which clients receive an incoming message. A client send one or more command messages (below) to the server to say what categories it is interested in. Each such subscription request contains a string. The string can be either plain text, or it can be a regular expression. The string is matched against the category, and if it does indeed match, then the message is passed on to the client.

Even if more than one subscription match, the incoming message is sent only once.

Client messages to the server

Client messages are mostly about handling subscriptions.

Here are examples of subscribe and unsubscribe messages, taken from the example program:

subscribe 1 regexp .  Send me all peer messages
subscribe 2 begins-with /upper  Send me all peer messages whose category starts with “/upper”
subscribe 3 ends-with shape  Send me all peer messages whose category ends with “shape”
subscribe 4 contains colour  Send me all peer messages whose category contains with “colour”
subscribe 5 is /upper/shape  Send me all peer messages whose category is exactly “/upper/shape”

The regexp (regular expressions) example will match every message, since a category is at least one character long. Regular expressions are not anchored. There is no implicit ^ at the beinning or $ at the end.

Server messages to a client

Client message messages are ususally send in response to a command.

The test program

Everybody loves screen dumps:

The test program display two geometric figures, the upper figure and the lower figure. Each figure has two properties, its shape and its colour. Shapes are circle, square and triangle. Colours are red, green and blue. When the user clicks one of the shape or colour buttons, a peer message is sent to the message bus. Subscribing clients receive the message. Hopefully they can interpret it, but the message bus does not care; once the message has been delivered its job is done.

The program is its own client. It does not update the figures when a button is pressed. Instead, it sends a peer message to the message bus, and if it has a matching subscription it will get the message back from the message bus. Only then will it update the figures. This design simplifies the program, and makes it possible to test the message bus using a single instance of the test program.

When the program starts, it has no subscriptions, and the figures do not get updated. Click one or more of the subscription buttons to get all or some of the peer messages.

If a new client is started, click the Send all button on an old instance. This will send the four messages needed by the new client to get synchronized.

There is no way that the server could handle this initialization. It does not know which messages are needed. Millions of messages may have passed through the server since the relevant message, so adding a message history, and sending it to newly connected clients, will not help.

One way to solve the problem is to implement a special message sent by clients when they connect. Then one or a few, but not all, clients can listen for, and reply to, this special message. But what happens if those special clients go away? Building a robust infrastructure is not trivial, and beyond the scope of this article.

Processing subscription messages

When the server receives a peer message, it will match the category of the message to each of the subscriptions. Every client that has one or more matching subscriptions will get the message once; duplicates due to multiple matching subscriptions are supressed. This supression applies only to the current peer message. The server does not remember old messages. If an identical peer message arrives, it will be delivered.

telnet as a client

The simplest possible message bus client is telnet:

$ telnet localhost 4711
s 1 r .

The command s 1 r . is short for subscribe 1 regexp .

This subscribes to all messages. It works whatever convention is used for categories, since a category is always at least one character long.

Using telnet you can send messages not available in the test program, such as:

:/upper/colour #666

Remember the leading colon.

Download the source

The message bus implementation and the test program is available for download.

Unpacking the archive will give you a directory message-bus with the source in it. Typing make will give you an binary called ./i386/gcc/debug/bin/message-bus.elf. The server, when started, will bind to the first available port starting with 4711.

Typing ./message-bus-tester.tcl will start the client. It will try ports 4711 to 4720.

Normally, the server will always start on port 4711, but if it crashes it will under certain circumstances leave that port in use, even though the program has exited. The operating system will detect that in a minute or two, but until then port remains in use, and can’t be used by the new instance of the server. This is a situation that everyone who has done low-level server programming recognize.

You can reach me by email at “lars dash 7 at sdu dot se” or by telephone +46 705 189090

View source for the content of this page.