pyroute2 as a messaging system

It is possible to use pyroute2 library as a messaging system. Initially it was written to provide a remote netlink access, but occasionally it evolved into a simple messaging system, that supports REP/REQ, PUSH/PULL and SUBSCRIBE models. Supported transports are:

• raw TCP: tcp://host:port
• SSLv3: ssl://host:port
• TLSv1: tls://host:port
• UDP: udp://host:port – only for PUSH/PULL
• raw UNIX sockets: unix://path
• UNIX + SSLv3: unix+ssl://path
• UNIX + TLSv1: unix+tls://path

Note

On Linux, it is possible to use the abstract socket namespace. To create such connection, use unix://\x00path notation.

Note

It is possible to use both absolute and relative paths for UNIX sockets: unix:///an/absolute/path and unix://a/relative/path

Note

UDP sockets can be used only for PUSH/PULL model.

Unlike other messaging systems, it is totally up to developer, which model to use: there is no socket types.

All examples will be done with IPRoute class, but all the some is true for all Netlink-based classes.

REP/REQ model

Server:

from pyroute2 import IPRoute

ipr = IPRoute()
ipr.serve('tcp://localhost:9824')
...  # wait for some event to exit
ipr.release()

Client:

from pyroute2 import IPRoute

ipr = IPRoute(host='tcp://localhost:9824')
print(ipr.get_addr())
ipr.release()

PUSH/PULL model

Server gets events from client and print them:

from pyroute2 import IPRoute

ipr = IPRoute(do_connect=False)
ipr.serve('udp://localhost:9824')
ipr.monitor()

while True:
    msg = ipr.get(raw=True)  # raw=True -- don't wait NLMSG_DONE
    print(msg)

ip.release()

Client pushes messages:

from pyroute2 import IPRoute

ipr = IPRoute()
addr = ipr.connect('udp://localhost:9824')

for msg in ipr.get_addr():
    ipr.nlm_push(msg, realm=addr)

ipr.disconnect(addr)
ipr.release()

SUBSCRIBE

Server provides events from OS level:

from pyroute2 import IPRoute

ipr = IPRoute()
ipr.serve('ssl://localhost:9824',
          key='server.key',
          cert='server.crt',
          ca='ca.crt')
...  # wait for some event to exit
ipr.release()

Client monitors events:

from pyroute2 import IPRoute

ipr = IPRoute(host='ssl://localhost:9824',
              key='client.key',
              cert='client.crt',
              ca='ca.crt')
while True:
    msg = ipr.get()
    print(msg)

ipr.release()

Building blocks

General scheme:

API (client) <--> Broker <--> Broker <-...
                    ^
                    |
                    v
              Target service

Brokers:

• Route packets
• Buffer packets and manage channels bandwidth
• Tag/untag packets
• Manage connections with other brokers and services
• Provide management API

Clients:

• Encode/decode packets to/from services
• Tag/untag packets into/from the transport protocol
• Manage connected brokers via management API

Services:

• Get untagged packets from clients via brokers
• Issue responses
• Issue broadcasts

Transport protocol

All the packets between clients and brokers should be incapsulated into the transport protocol messages. Each transport message consists of:

struct nlhdr {
    uint32 length;
    uint16 type;
    uint16 flags;
    uint32 sequence_number;
    uint32 pid;
};

struct envmsghdr {
    uint32 dst;  /* target subsystem addr */
    uint32 src;  /* not used yet */
};

struct nla_hdr {
    uint16 length;
    uint16 type;  /* == 0; now it is the only NLA for envmsg */
};

Then follows binary data of incapsulated messages. Length type of uint16 limits data to 65535 bytes. Fragmentation and reassembling of larger messages is up to client and service; brokers have nothing to do with message reassembling.

Control protocol

Control messages go between clients and brokers incapsulated in the transport messages; flags field of nlhdr should be set to 1.

Control message format:

struct nlhdr {
    uint32 length;
    uint16 type;
    uint16 flags;
    uint32 sequence_number;
    uint32 pid;
};

struct genlmsghdr {
    uint8 cmd;
    uint8 version;
    uint16 reserved;
}

array of NLAs