Version2¶

Description¶

This page describes the Cyclone DDS Python binding that can be found at src/demonstrators/PythonBinding/poc2. This package is wrapper on Cyclone’s C library, with the aim of providing access to all functionalities, with a pythonic API. It was designed to fulfill a number of requirements:

Completeness: All the functionalities that are implemented within Cyclone DDS should be available

from the Python version.
DDS wire-standard: The current version developed by atolab uses pre-defined topics, by converting

everything to a string, regardless of the actula data type of the field. this makes it impossible to communicate with different implementations. This library should obey the DDS wire-standard.
Pythonic: The implementation should be done in a Pythonic way, and it should feel natural for the

Python developer.

Warning

The library is not yet fully implemented. It does not support sequence fields, and nested topic types, and it is not possible to specify quality of service options. These will be implemented soon.

The role of IDL¶

In order to understand how the Python binding deals with defining and registering topic types, we give a brief overview of how this is traditionally (both in the C version, as well as in the atolab Python version) approached.

DDS is a topic based protocol. A topic is an abstract entity, that assigns both semantic and syntactic meaning to the bytes that are being sent over the network. It ensures that the receiver can decode the message properly, and also provides a mean of error checking for the sender. There are however a number of other important attributes of a topic other than its byte layout. For example specifying quality of service options, and setting different configuration flags are topic level operations, that need to be stored in the global data space, alongside with the topic definition. There is also an important distinction between keyed and keyless topics: Having no key specified makes all the instances of a topic effectively equal (meaning there is no distinction possible between the instances). So when a field is set to be a topic key, this information needs to be known by the golbal data space, that stores the messages and manages queues.

To be able to do all this, it is not sufficent to simply define the structure (in the C sense) of the desired topic, but it also becomes necessary to provide some sort of configuration object, that describes the aforementioned attributes. This configuration object (which is not really an object itself, but a C-style structure) is called the topic descriptor. To make life easier for the developer, Cyclone DDS lets the user define the topic structures in a tiny domain specific language, called IDL. A special parser reads this definition, and generates the corresponding C data types for both the topic structure and the topic descriptor. These generated C files can then be used to implement the program.

How to be Pythonic?¶

This approach with the IDL file works alright when working with C, or any other compiled language, because the use of an additional static tool to get your topic generated is not of huge inconvenience. When using a script language like Python, it becomes much more of a problem. It suddenly becomes a lot more annoying to let a third-party tool generate part of your code every time you change something. So it is necessary to find a solution that lets the user circumvent the use of IDL definitions.

In the atolab Python library, they decided to get rid of the possibility of defining your own topic altogether. Instead, they limited the user to a single topic (for which IDL is still used by the way), and every possible value that can be sent is reduced to this single topic type. It seems ok on the surface, as there is no need to deal with the static definition of topic types, but there are two major problems with this approach. Firstly, to ensure that everything can be reduced to a given topic, it is chosen to have a string field that esentially contains all the data. Every time you send something over the network, the library converts the data into a json string. This is of course extremely inefficient. Not only does it cost time and computational resources to convert every single sample into a string, but it also increases the network load by a lot. For example, if you want to send the number 3658, it will not be sent as a singe integer, but as four different characters. Another, more obvious problem with this approach is that it entirely disobeys the DDS wire-standard, meaning it is only able to communicate with another atolab client, but not with a different DDS implementation.

At the start of the project there were two possible options for tackling this issue: We can either implement our own IDL parser, that outputs Python run-time objects instead of .c/.h files, or that we leave out the static definition entirely, and do all things in Python. After discussing this issue with the product owner [Albert Mietus] on several occasions, we settled on the second approach, because of being more modern and Pythonic.

Design¶

Domain¶

Participants in DDS are all part of a domain. A domain roughly means that every endpoint that is on domain x can only communicate with other endpoints on domain x, and not with others. Domains are to separate different networks of devices from eachother. When creating an application, first you must specify the domain of it, with a domain ID.

domain = Domain(1)

Topic¶

The goal is to define topic structures as ordinary Python classes. In Python, everything is dynamically typed, which means that when a class is defined, it is not clear what the types of its attributes are going to be (and those types do also not necessarily remain the same throughout the execution of the program). This behaviour makes it impossible to generate the proper topic descriptor when creating a new topic. To make it possible we need to introduce a way of fixing the type of the attributes when defining a class. One way of doing this is having a static dictionary, or list of tuples that maps each attrinute to their correesponding type. Then we would get something like this:

class Data:
     _fields_ = [
         "id": int,
         "name": str
     ]

     def __init__(self):
         self.id = None
         self.name = None

It describes the topic _Data_, that has two attributes, id and name. This is however not the most elegant solution, as the user has to add a class level attribute, fields, which makes the code less readable, and annoying to write it out if there are many attributes. It is also inconvenient having to initialize them, as their value will be determined run-time. (If you could initialize any of your fields as a contant, it would make no sense to send it over the network, as you could initialize it at the other side to the same constant as well.) Since everything is an object in Python, even types, we chose for a solution that mitigates both issues, and feels a lot more Pythonic. You would then define the class above like this:

class Data:
   def __init__(self):
      self.id   = int
      self.name = str

To turn this into a proper topic, the C struct of this class, and its topic descriptor needs to be generated and passed to DDS. This work is done in the topic class decorator.

Note

The _topic_ decorater makes run-time changes on the class definition, ads new attributes to it, and most importantly replaces the __init__ function. It is important that the original __init__ does not have any arguments other than self, and optionally **kwargs.

As an extra argument, the decorator also takes the domain on which this topic will run.

domain = Domain(1)   # '1' is the domain id.

@topic(domain)
class Data:
   def __init__(self):
      self.id   = int
      self.name = str

The topic “Data” is now properly registered by Cyclone, and instances of it can now be sent over the network. It can be instantiated by supplying it with keyword argumets for the attributes.

data = Data(name="Kevin", id=27)

It is not necessary to initialize all of the fields. There is a default value associated with each type. The uninitialized fields take the default value.

>>> data = Data(name="Jan")
>>> data.id
>>> 0

It is also possible to be more specific with typing. There are a number of C types included in the library, such as int16, int32, uint16 etc.

Sending and receiving¶

You can publish using a Writer object. The constructor of writer takes a topic that it can write to, and can be used by calling its __call__ method with the data sample. Below is a minimalistic example of a program that writes a single data point:

domain = Domain(1)       # '1' is the domain id.

@topic(domain)
class Data:
   def __init__(self):
      self.id   = int
      self.name = str

w = Writer(Data)         # Creating writer object.

sample = Data(name="Jaap", id=12)
w(sample)                # Writing the sample data

Reading from a topic can be done with the Reader class. Similarily to Writer, when creating a Reader you need to pass it the topic. Additionally you also need to specify the buffer size. It is the number of elements that can be stored unread. So it is the size of the queue for unread messages. A call to Reader.__call__ returns a list of data points, flushing the buffer. Here is an example of the “other half” of our program:

domain = Domain(1)       # '1' is the domain id.

@topic(domain)
class Data:
   def __init__(self):
      self.id   = int
      self.name = str

r = Reader(Data, 10)     # Creating reader object.
samples = r()            # samples now contains a (possibly empty) list of Data objects.