DDS Performance Study

In this part we look at important design choices around DDS and discover how different choices influence the performance of DDS. The design choices in question can be summarized to the following:

1. Docker deployment and its effect on performance.
2. Ethernet vs. WiFi networking.
3. Router effects on performance.
4. CPU frequency scaling and its effect on performance.

In order to answer all these questions we choose the Roundtrip and Flood demonstrators since the former gives a good estimate on how quickly a message can move through our DDS network and the latter also saturates our network as well as the processor usage. In each case we look at a network of 4 hubs with 1 master hub and 3 slave hubs and a Flood with size of 1000 messages. For more information on the setup refer to Round trip and Flood in C.

Docker and Ethernet vs. WiFi Performance

In this section we are going to look at the performance of DDS when deployed using standalone executables on a machine compared to deployment of DDS through containers. In each scenario we also look at both wired and wireless networking. We define the following scenarios:

1. 1 Node: all 4 hubs run on a single machine (Inter-Process Communication).
2. Docker 1 Container: all 4 hubs run inside one container on one machine (Inter-Container Communication).
3. 2 Nodes: Master hub runs on machine A, Hub 1 runs on Machine B, Hub 2 runs on machine A and Hub 3 runs on machine B (Inter-System Communication).
4. Docker 2 Containers: Master hub runs in container 1A of machine A, Hub 1 runs in container 1B of Machine B, Hub 2 runs in container 1A of machine A and Hub 3 runs in container 1B of machine B.
5. Docker 4 Containers: Master hub runs in container 1A of machine A, Hub 1 runs in container 1B of Machine B, Hub 2 runs in container 2A of machine A and Hub 3 runs in container 2B of machine B.

The results for both the Roundtrip and Flood experiments can be seen in the figures above. We can conclude that there is no performance penalty for using DDS inside a Docker container, this can be seen by comparing the 1 Node to the Docker 1 Container for both demonstrators. Furthermore, we can deduct that over Ethernet Docker container deployment performs in an identical manner to the normal standalone deployment (i.e. comparing 2 Nodes to Docker 2 Containers). This is an expected result since we use host networking which means that the container sees the host’s network as its own. In the case we would use bridged networking mode we would expect the performance to be worse due to the added overhead of going through the host’s internal bridge connection. In addition, we also see that for the Flood experiment over Ethernet all experiments yield the same results due to the fact that we saturate our network bandwidth. In the case of WiFi, it is logical to have larger execution times when moving to 2 separate nodes.

In conclusion, we expected the wireless setup to perform much slower than a wired connection. The most important take away from this study is that Docker containers perform as good as a native implementation. More on this matter can be found in an interesting paper by IBM with the title`An Updated Performance Comparison of Virtual Machines and Linux Containers <https://tinyurl.com/hmj4vex>`_. Finally, we note that inter-container communication results in the same connection penalty as communication between separate nodes.

Router Effect on Performance

In this section, we look at the effect different routers will have on the performance. This research is conducted in order to determine the difference between the following categories of routers:

1. Cheap Embedded Router Solution, such as the Raspberry Pi Zero W.
2. Modular lightweight and low cost Solution, such as a USB WiFi dongle.
3. Dedicated mid price point router Solution.
4. Dedicated high price point router Solution.

The routers under investigation are the following:

1. Raspberry Pi Zero W.
2. Asus USB-AC53 dongle.
3. Linksys X3500 router.
4. Arcadyan AC2200 router.

Both Roundtrip and Flood demonstrators were setup up with the 2 node configuration:

• 2 Nodes: Master hub runs on machine A, Hub 1 runs on Machine B, Hub 2

  runs on machine A and Hub 3 runs on machine B (Inter-System Communication).

It is important to note at this point that all configurations were positioned in close proximity from the wireless router in order to alleviate any differences caused by different antenna sizes at higher distances (i.e. dedicated routers larger antenna arrays). The results show a clear advantage for the dedicated routers. In the case of the Roundtrip demo we see that the expensive router provides a 2 time performance boost over the cheap embedded router in the Raspberry Pi Zero W, while in the case of Flood where the network is saturated this difference is in the order of 3. We can already draw our first conclusion, which is that for applications where higher bandwidth is required, a dedicated router is the clear winner. At the same time, if we look at the cost effectiveness of our solution the price difference is almost 10x as high for the expensive dedicated router over a Raspberry Pi Zero. Similarly, the USB WiFi dongle also gives a performance boost over the embedded router but the difference in performance does not justify the difference in price which is 2x more just for a dongle compared to a full computer. Finally, it is also interesting to compare the two dedicated routers that come at different price points (50 euros vs. 100 euros). The more expensive router is by a small margin faster. It is interesting to note that the two dedicated routers were also tested in the same experimental setup over Ethernet connection, with identical performance results. In conclusion, the choice of router really depends on the application. An application that requires high bandwidth and/or across an increasing number of nodes will definitely benefit vastly from a dedicated high performance router. On the other hand, one can easily see the benefits of using an embedded device or a dongle as a router.

Effect of CPU Frequency Scaling on Performance

In this section we investigate the effect of a CPU frequency scaling on the performance of our two DDS demonstrators (i.e. Roundtrip and Flood). For this setup we choose to utilize the Inter-process Communication on a single machine. Furthermore, we choose to utilize the Sogeti laptop that utilizes an Intel i5-8350U with 4 cores and 8 threads. This choice was made mainly due to the ease of scaling the frequency on a Windows machine through different maximum CPU states.

The results show an exponential decay in execution time with respect to CPU frequency, for both Roundtrip and Flood. As a result, we can conclude that Flood is bandwidth bound and the communication speed is more important than the per node calculation time.