Disclaimer! I am in no way an expert at this. I’m learning it as part of a research project at work. This will be the first post in what I hope to be a series of posts on TSN.

What is it?

Time-Sensitive Networking (TSN) is a topic in the networking world where there exists data frames that absolutely must arrive within a limited time. It is a fairly new topic and some of the techniques are not completely industry-tested yet. It originates from the sound and video world where they have sound and video that must arrive very quickly. It was originally called Audio Video Bridging (AVB), but was renamed to Time-Sensitive Networking (TSN) in 2012.

It could be used at concerts, for example. The sound data must to travel from the microphones to all the speakers, and maybe some cameras to stream to the screen; and it all has to happen very quickly. Humans are very sensitive to delays in sound and video when it doesn’t match up with the reality. To make this even more complex there is often other types of data travelling the network as well. You could make the argument that the sound should use its own cables; however, that is extra maintenance and requires a separate network to run on. It is convenient to be able to use the same network for everything.

To work around this they have come up with several techniques for handling Quality of Service (QoS) in networks. I will cover some QoS techniques now, and future posts will go more in-depth on some of them. These are all defined in IEEE standards.

TSN techniques

Strict Priority (802.1Q)

The most basic form of QoS. This, as well as several of the below techniques, are based on the concept of multiple egress queues. Usually 8 of them. With 8 queues you can do a 1-to-1 mapping from the Priority Code Point (PCP) in the VLAN tag. There are some techniques to set the priority from software (see man mqprio). In Linux, this is the skb->priority field. But when you use QoS you usually want the frames to only pass through the hardware. There is currently only support in the Linux kernel to set this for this for the DSCP field (man dcb-app), but hopefully we can get a proper way to set this for PCP as well. It is already supported by a lot of the hardware. There are other ways to do this in the hardware as well, but not all are ideal.

As for the algorithm itself, it is very simple. It transmits frames in order of priority. When it starts transmitting a frames it picks from the highest priority queue that is not empty. A lower priority queue will only transmit if all higher priority queues are empty. Below is an illustration of 3 transmit queues with frames A-H, where frames in queue 2 would have the highest priority.

Queue
    ----------------------
  0 ->     [H] [G] [F] [D] ->
    ----------------------
  1 ->             [E] [C] ->
    ----------------------
  2 ->             [B] [A] ->
    ----------------------

Time-Aware Shaper (802.1Qbv)

This extends the Strict Priority algorithm by adding a time aspect to it and splitting the time into several slots where different priorities are allowed to send in each slot. For example, it could have 2 slots, one where only priority 7 (highest) is allowed to send, and another where priority 0-6 are allowed to send. The time slots can be of different length. For the second slot there are 7 different priorities that can send, so between those it applies Strict Priority.

Time-Aware Shapers can run in software, but the benefit it gives is completely lost. To be at all useful it needs to be offloaded to hardware, which also requires a PTP hardware clock. On top of the clock itself it should also be time synced to all other switches in the network. Otherwise, there is also very little point in using it. If all switches are synced it will result in the same priorities all being open at once in the network; ideally letting the frames flow all the way to their destination without being interrupted by other traffic. To put it simply, it can reserve bandwidth for the different priorities and giving the network a deterministic behavior (at least for high priorities), but of course this assumes you aren’t overloading the network itself. If you try to overload the link it will obviously drop and delay frames.

[Read more…]({% post_url networks/2022-08-25-time-aware-shaper %})

Credit-Based Shaper (802.1Qav)

The original TSN technique that was invented for the purpose of audio and video. It reduces the congestion in the network by evening out bursts. It is a form of fair queueing. Like Strict Priority, this also uses multiple egress queues. Each queue holds a credit score that goes up when a queue is not transmitting, and down when it transmits. When the credit score goes negative it is not allowed to transmit. By evening out the traffic to get rid of bursts the data can flow more freely in the network. This is especially useful when there are many devices in the network that may send bursts. If all send a burst at once it can result in the queues filling up where the bursts meet.

Frame Preemption (802.1Qbu)

Frame preemption is the only of the priority-based TSN techniques that requires both ends of a link to support its protocol. The ones above can all operate on a single switch, though ideally used in a network were all are running the same technique. It also requires hardware support. With frame preemption both sides work together to provide faster delivery speed. To begin, we first have to define which priorities (queues) are to be considered preemptible. This is done independently on each switch.

If a preemptible frame is currently being transmitted and an express (non-preemptible) frame arrives to the queue it will almost immediately stop the preemptible frame (it preempts it). To not have wasted all the work already done sending the frame it appends a CRC32 checksum to the end of the partial frame, as well as inverts the last 16 bits of the CRC value to distinguish a partial frame from a regular frame. When that is done it can start sending the express frame. The partial frame is stored in a separate buffer on the other end and waits for the other half of its frame to arrive. The other half is sent when no more express traffic is available to send.

Frame Replication and Elimination for Reliability (802.1CB)

The final technique for this post is the only one not dependent on multiple queues and priorities. Rather than providing fast delivery it is a form of redundancy. The idea is that it duplicates frames and sends them different paths in a ring or otherwise redundant topology. In the case of one switch failing it doesn’t even have to wait for the switches to learn the new path. The frames are already on their way the other path as well. To not get duplicate frames at the destination it eliminates any duplicates where they meet again, ideally as close to the destination as possible; possibly even at the end device. Same as frame preemption, this requires the mergeing device to also support the protocol. The intermediary switches do not need to support it, they just need to forward the frame.

Keeping track of duplicates in the merging device is done with the help of a sequence number in the duplicated frames. This is called the R-tag and it has a 16-bits field to store the sequence number. The R-tag is then removed when a frame arrives at the merging device, and any following frames with the same sequence numbers are discarded.

In the case of multiple senders to the same merging device it can employ a stream identification function to identify which data stream a frame belongs to. Two frames can have the same sequence number but belong to different streams, and we don’t want one of those to be discarded because they are not the same frame.

TSN today

TSN techniques are still considered fairly new and is not entirely industry-proven. It is starting to appear more and more in factories with the advent of Industry 4.0 and companies wanting to make their factories smart and more modern. By using a single network for everything you simplify connections and reduce the amount of cables. The time-sensitive aspect comes from, for example, control systems that need to send instructions instantly other machines. It has also gained traction in the automobile industry for use inside cars to connect various smart devices such as cameras, sensors, control devices, etc., with the main computers.