Developer’s guide to Netlink and pyroute2

Netlink basics

General netlink packet structure:

nlmsg packet:
    + header
    + data

Generic netlink message header:

nlmsg header:
    + uint32 length
    + uint16 type
    + uint16 flags
    + uint32 sequence number
    + uint32 pid

The length field is the length of all the packet, including data and header. The type field is used to distinguish different message types, commands etc. Please note, that there is no explicit protocol field – you choose a netlink protocol, when you create a socket.

The sequence number is very important. Netlink is an asynchronous protocol – it means, that the packet order doesn’t matter and is not guaranteed. But responses to a request are always marked with the same sequence number, so you can treat it as a cookie.

Please keep in mind, that a netlink request can initiate a cascade of events, and netlink messages from these events can carry sequence number == 0. E.g., it is so when you remove a primary IP addr from an interface, when promote_secondaries sysctl is set.

Beside of incapsulated headers and other protocol-specific data, netlink messages can carry NLA (netlink attributes). NLA structure is as follows:

NLA header:
    + uint16 length
    + uint16 type
NLA data:
    + data-specific struct
    # optional:
    + NLA
    + NLA
    + ...

So, NLA structures can be nested, forming a tree.

Complete structure of a netlink packet:

nlmsg header:
    + uint32 length
    + uint16 type
    + uint16 flags
    + uint32 sequence number
    + uint32 pid
[ optional protocol-specific data ]
[ optional NLA tree ]

More information about netlink protocol you can find in the man pages.

Pyroute2 and Netlink

packets

To simplify the development, pyroute2 provides an easy way to describe packet structure. As an example, you can take the ifaddrmsg description – pyroute2/netlink/rtnl/ifaddrmsg.py.

To describe a packet, you need to inherit from nlmsg class:

from pyroute2.netlink import nlmsg

class foo_msg(nlmsg):
    fields = ( ... )
    nla_map = ( ... )

NLA are described in the same way, but the parent class should be nla, instead of nlmsg. And yes, it is important to use the proper parent class – it affects the header structure.

fields attribute

The fields attribute describes the structure of the protocol-specific data. It is a tuple of tuples, where each member contains a field name and its data format.

Field data format should be specified as for Python struct module. E.g., ifaddrmsg structure:

ifaddrmsg structure:
    + unsigned char ifa_family
    + unsigned char ifa_prefixlen
    + unsigned char ifa_flags
    + unsigned char ifa_scope
    + int ifa_index

should be described as follows:

class ifaddrmsg(nlmsg):
    fields = (('family', 'B'),
              ('prefixlen', 'B'),
              ('flags', 'B'),
              ('scope', 'B'),
              ('index', 'I'))

Format strings are passed directly to the struct module, so you can use all the notations like >I, 16s etc. All fields are parsed from the stream separately, so if you want to explicitly fix alignemt, as if it were C struct, use the pack attribute:

class tstats(nla):
    pack = 'struct'
    fields = (('version', 'H'),
              ('ac_exitcode', 'I'),
              ('ac_flag', 'B'),
              ...)

Explicit padding bytes also can be used, when struct packing doesn’t work well:

class ipq_mode_msg(nlmsg):
    pack = 'struct'
    fields = (('value', 'B'),
              ('__pad', '7x'),
              ('range', 'I'),
              ('__pad', '12x'))

nla_map attribute

The nla_map attribute is a tuple of NLA descriptions. Each description is also a tuple in two different forms: either two fields, name and format, or three fields: type, name and format.

Please notice, that the format field is a string name of corresponding NLA class:

class ifaddrmsg(nlmsg):
    ...
    nla_map = (('IFA_UNSPEC',  'hex'),
               ('IFA_ADDRESS', 'ipaddr'),
               ('IFA_LOCAL', 'ipaddr'),
               ...)

This code will create mapping, where IFA_ADDRESS NLA will be of type 1 and IFA_LOCAL – of type 2, etc. Both NLA will be decoded as IP addresses (class ipaddr). IFA_UNSPEC will be of type 0, and if it will be in the NLA tree, it will be just dumped in hex.

NLA class names are should be specified as strings, since they are resolved in runtime.

There are several pre-defined NLA types, that you will get with nla class:

• none # forces pyroute2 just to skip this NLA
• uint8
• uint16
• uint32 # there are dedicated NLA of these types as well
• ipaddr # IP address, IPv4 or IPv6, depending on the socket
• l2addr # MAC address
• hex # hex dump as a string – useful for debugging
• cdata # just a binary string
• asciiz # zero-terminated ASCII string

Please refer to pyroute2/netlink/__init__.py for details.

You can also make your own NLA descriptions:

class ifaddrmsg(nlmsg):
    ...
    nla_map = (...
               ('IFA_CACHEINFO', 'cacheinfo'),
               ...)

    class cacheinfo(nla):
        fields = (('ifa_prefered', 'I'),
                  ('ifa_valid', 'I'),
                  ('cstamp', 'I'),
                  ('tstamp', 'I'))

Custom NLA descriptions should be defined in the same class, where they are used.

Also, it is possible to use not autogenerated type numbers, as for ifaddrmsg, but specify them explicitly:

class iw_event(nla):
    ...
    nla_map = ((0x8B00, 'SIOCSIWCOMMIT', 'hex'),
               (0x8B01, 'SIOCGIWNAME', 'hex'),
               (0x8B02, 'SIOCSIWNWID', 'hex'),
               (0x8B03, 'SIOCGIWNWID', 'hex'),
               ...)

Here you can see custom NLA type numbers – 0x8B00, 0x8B01 etc. It is not permitted to mix these two forms in one class: you should use ether autogenerated type numbers (two fields tuples), or explicit numbers (three fields typles).

parsed netlink message

Netlink messages are represented by pyroute2 as dictionaries as follows:

{'header': {'pid': ...,
            'length: ...,
            'flags': ...,
            'error': None,  # if you are lucky
            'type': ...,
            'sequence_number': ...},

 # fields attributes
 'field_name1': value,
 ...
 'field_nameX': value,

 # nla tree
 'attrs': [['NLA_NAME1', value],
           ...
           ['NLA_NAMEX', value],
           ['NLA_NAMEY', {'field_name1': value,
                          ...
                          'field_nameX': value,
                          'attrs': [['NLA_NAME.... ]]}]]}

As an example, a message from the wireless subsystem about new scan event:

{'index': 4,
 'family': 0,
 '__align': 0,
 'header': {'pid': 0,
            'length': 64,
            'flags': 0,
            'error': None,
            'type': 16,
            'sequence_number': 0},
 'flags': 69699,
 'ifi_type': 1,
 'event': 'RTM_NEWLINK',
 'change': 0,
 'attrs': [['IFLA_IFNAME', 'wlp3s0'],
           ['IFLA_WIRELESS',
            {'attrs': [['SIOCGIWSCAN',
                        '00:00:00:00:00:00:00:00:00:00:00:00']]}]]}

create and send messages

Using high-level interfaces like IPRoute or IPDB, you will never need to manually construct and send netlink messages. But in the case you really need it, it is simple as well.

Having a description class, like ifaddrmsg from above, you need to:

• instantiate it
• fill the fields
• encode the packet
• send the encoded data

The code:

from pyroute2.netlink import NLM_F_REQUEST
from pyroute2.netlink import NLM_F_ACK
from pyroute2.netlink import NLM_F_CREATE
from pyroute2.netlink import NLM_F_EXCL
from pyroute2.iproute import RTM_NEWADDR
from pyroute2.netlink.rtnl.ifaddrmsg import ifaddrmsg

##
# add an addr to an interface
#

# create the message
msg = ifaddrmsg()

# fill the protocol-specific fields
msg['index'] = index  # index of the interface
msg['family'] = AF_INET  # address family
msg['prefixlen'] = 24  # the address mask
msg['scope'] = scope  # see /etc/iproute2/rt_scopes

# attach NLA -- it MUST be a list / mutable
msg['attrs'] = [['IFA_LOCAL', '192.168.0.1'],
                ['IFA_ADDRESS', '192.162.0.1']]

# fill generic netlink fields
msg['header']['sequence_number'] = nonce  # an unique seq number
msg['header']['pid'] = os.getpid()
msg['header']['type'] = RTM_NEWADDR
msg['header']['flags'] = NLM_F_REQUEST |\
                         NLM_F_ACK |\
                         NLM_F_CREATE |\
                         NLM_F_EXCL

# encode the packet
msg.encode()

# send the buffer
nlsock.sendto(msg.buf.getvalue(), (0, 0))

Please notice, that NLA list MUST be mutable.

---

Module contents:

exception pyroute2.netlink.NetlinkDataDecodeError(exception)

The error occured while decoding the message fields

exception pyroute2.netlink.NetlinkDecodeError(exception)

Base decoding error class.

Incapsulates underlying error for the following analysis

exception pyroute2.netlink.NetlinkError(code, msg=None)

Base netlink error

exception pyroute2.netlink.NetlinkHeaderDecodeError(exception)

The error occured while decoding a header

exception pyroute2.netlink.NetlinkNLADecodeError(exception)

The error occured while decoding NLA chain

class pyroute2.netlink.cdatamsg(buf=None, length=None, parent=None, debug=False)

Simplest netlink CDATA container

class pyroute2.netlink.ctrlmsg(buf=None, length=None, parent=None, debug=False)

Netlink control message

class pyroute2.netlink.genlmsg(buf=None, length=None, parent=None, debug=False)

Generic netlink message

class pyroute2.netlink.mgmtmsg(buf=None, length=None, parent=None, debug=False)

Messaging system control messages

class pyroute2.netlink.nla(buf=None, length=None, parent=None, debug=False)

Main NLA class

class pyroute2.netlink.nlmsg(buf=None, length=None, parent=None, debug=False)

Main netlink message class

header

alias of nlmsg_header

class pyroute2.netlink.nlmsg_atoms(buf=None, length=None, parent=None, debug=False)

A collection of base NLA types

class asciiz(buf=None, length=None, parent=None, debug=False)

Zero-terminated string.

class nlmsg_atoms.cdata(buf=None, length=None, parent=None, debug=False)

Binary data

class nlmsg_atoms.hex(buf=None, length=None, parent=None, debug=False)

Represent NLA’s content with header as hex string.

class nlmsg_atoms.ipaddr(buf=None, length=None, parent=None, debug=False)

This class is used to decode IP addresses according to the family. Socket library currently supports only two families, AF_INET and AF_INET6.

We do not specify here the string size, it will be calculated in runtime.

class nlmsg_atoms.l2addr(buf=None, length=None, parent=None, debug=False)

Decode MAC address.

class nlmsg_atoms.none(buf=None, length=None, parent=None, debug=False)

‘none’ type is used to skip decoding of NLA. You can also use ‘hex’ type to dump NLA’s content.

class pyroute2.netlink.nlmsg_base(buf=None, length=None, parent=None, debug=False)

Netlink base class. You do not need to inherit it directly, unless you’re inventing completely new protocol structure.

Use nlmsg or nla classes.

get_attr(attr, default=None, fmt='raw')

Return the first attr by name or None

get_attrs(attr, fmt='raw')

Return attrs by name

get_encoded(attr, default=None)

Return the first encoded NLA by name

getvalue()

Atomic NLAs return their value in the ‘value’ field, not as a dictionary. Complex NLAs return whole dictionary.

classmethod name2nla(name)

Convert human-friendly name into NLA name

Example: address -> IFLA_ADDRESS

Requires self.prefix to be set

classmethod nla2name(name)

Convert NLA name into human-friendly name

Example: IFLA_ADDRESS -> address

Requires self.prefix to be set

register_nlas()

Convert ‘nla_map’ tuple into two dictionaries for mapping and reverse mapping of NLA types.

ex: given::

nla_map = ((‘TCA_HTB_UNSPEC’, ‘none’),

(‘TCA_HTB_PARMS’, ‘htb_parms’), (‘TCA_HTB_INIT’, ‘htb_glob’))

creates::

t_nla_map = {0: (<class ‘pyroute2...none’>, ‘TCA_HTB_UNSPEC’),

1: (<class ‘pyroute2...htb_parms’>, ‘TCA_HTB_PARMS’), 2: (<class ‘pyroute2...htb_glob’>, ‘TCA_HTB_INIT’)}

r_nla_map = {‘TCA_HTB_UNSPEC’: (<class ‘pyroute2...none’>, 0),

‘TCA_HTB_PARMS’: (<class ‘pyroute2...htb_parms’>, 1), ‘TCA_HTB_INIT’: (<class ‘pyroute2...htb_glob’>, 2)}

reserve()

Reserve space in the buffer for data. This can be used to skip encoding of the header until some fields will be known.

class pyroute2.netlink.nlmsg_header(buf=None, length=None, parent=None, debug=False)

Common netlink message header