UDP losses classification
In developing the distributed software system it might be reasonable to use exchange data over the UDP protocol to minimize exchange time. UDP is quite a simple way to act fast. But this simplicity comes with a lack of guarantee of data being delivered.
There are several sporadic factors of data stream packets loss:
- network media errors
- network-related hardware overloads
- source host-related hardware overloads
Media related losses
Network media errors come with the inability of the media to deliver data stream packets on specific sporadic circumstances. For example: on a magnetic storm at the north pole wi-fi media can be useless for several hours.
Network hardware overload
On network-related hardware overloads, packets can be lost in the case of data streams processing priorities. Network software processes the packets in a “priority queues algorithm”. That means that the most valuable child queue packets will be processed first of all child queues. If the UDP queue has no first priority, then packets are not sent on overload. New incoming UDP packets will make the queue bigger at the same time. The UDP queue will grow over the limit after some overload process duration.
On new UDP packets incoming on an over-limited child queue buffer - some old packets need to be dropped without sending to emplace new packets.
UDP source host-related losses
On source host hardware overload it is also possible to detect UDP packets losing. The matter of these packets losing is equal to losing at intermediate network hardware. But one use case on source host UDP packets lost is specific. UDP queue buffer overflow will happen if we will put packets with rates greater than the rate of packets getting from the queue. It is possible to reproduce on Linux just by actions at one host.
Linux UDP protocol processing implemented over act with one buffer which contains a queue of all packets from all processes. On some system-dependent factors (number of UDP clients processes, incoming packet size, incoming packets rate) this buffer can be over-limited. To change UDP buffer size at Linux it is possible to use the command string:
sysctl -w net.core.rmem_max=<your value>
It is a case that is not widely discussed but can confuse a lot. To minimize the harm it is good to remember that if well-packed data will be transferred better over UDP. It is good to have the UDP stream be split into chunks close to MTU. Packing for smaller-sized packets at one be also preferable. Then it will cover Linux buffer entries as best as possible.
Another factor to work on is data compression. Compression logical way can reduce packet rate on UDP clients side. That is why it is important to have compression as an option for any UDP-related application under Linux.
Conclusion
As we can see it is simple things that can make your UDP user experience better. Please share your comments and suggestions as well!