QoS: priority lanes on a crowded highway

There's another traffic jam on the Packet Highway. After the last adventures with ping, jitter and packet loss, the manager has found a solution: everybody should drive faster. Unfortunately, the cars aren't moving. Only the complaint counter is increasing at high speed.

An ambulance carrying a patient is stuck in the left lane. Beside it sits a truck hauling 800 GB of cat videos. Behind them are a bus full of VoIP packets, a little SSH car, and a driver who's spent three hours downloading a Windows update.

The traffic manager makes an announcement:

— From now on, everybody is equal!
— But the ambulance has a patient!
— And the truck has cats!
— Boss, someone could die!
— Cats have feelings too!

The administrator puts down his tea.

— We need QoS.
— Another toll payment system?
— No. This time it's about keeping urgent packets from getting stuck behind less urgent ones.

QoS: traffic management, not magic

QoS — Quality of Service — is a collection of methods for classifying traffic, managing queues, allocating resources and controlling how a network behaves under congestion.

In our earlier story about ping, jitter and loss, we saw that bandwidth isn't everything. Packets may arrive late, irregularly or not at all. QoS helps decide what happens when the road narrows.

But QoS isn't a turbocharger. If a booth can process a thousand cars a minute, painting “QoS” above it won't increase that to ten thousand.

— Why bother, then?
— So the ambulance doesn't wait behind the cat truck.
— What about the cats?
— They'll arrive. Perhaps a little later.
— I'll consult the animal welfare regulations.

Classification: who's driving and what's on board?

A traffic officer inspects arriving vehicles.

— Ambulance: urgent.
— Voice call: sensitive to delay.
— System update: large delivery, can wait.
— Backup: important, but not necessarily this very second.

That's traffic classification. Network devices can classify packets based on addresses, ports, interfaces, VLANs and existing markings, among other things. A port number alone, however, doesn't reliably identify every application.

Traffic priority isn't the same as the business importance of the data. A backup may be essential to a company, but it doesn't necessarily need to jump ahead of a voice packet. The backup can wait a moment; that snippet of conversation will be useless if it arrives several seconds late.

DSCP: a sticker on the windscreen

The officer gets a new tool: colour-coded traffic stickers.

In IP networking, a similar role is played by DSCP — Differentiated Services Code Point, a six-bit field in the IP header used to identify service classes. Devices may interpret the marking and apply an appropriate queueing policy.

For example, EF (DSCP 46) is often used for voice traffic requiring low delay, while CS0 (DSCP 0) identifies default best-effort handling. But that's a network policy, not a universal right of way across the Internet.

Suddenly the cat-video truck arrives at the checkpoint displaying a huge sign: “EMERGENCY VEHICLE — TOP PRIORITY”.

— What are you carrying?
— Materials critical to the infrastructure.
— What kind?!
— A cat falling off a sofa. In 4K.
— Why have you marked yourself as an emergency vehicle?
— I gave myself priority.

🤣

And here's the catch: you cannot always trust markings supplied by a sender. At a trust boundary, an administrator may inspect, change or clear DSCP. If every customer could declare themselves an ambulance, the entire road would fill with ambulances — including three articulated lorries carrying cats.

Queues: who goes through the toll booth first?

Putting a label on a packet doesn't make it faster by itself. A device needs to act on the label.

The simplest FIFO queue works first in, first out. It sounds fair, until an ambulance is stuck behind hundreds of trucks.

With priority queueing, urgent traffic can be served before ordinary traffic. Weighted scheduling methods instead share capacity or service time between classes, helping avoid leaving one class completely forgotten.

— We'll build a special ambulance lane!
— Brilliant!
— But what if everyone drives in that lane?
— Then we're back where we started, except the paint cost more.

Unlimited priority can cause starvation: high-priority traffic continuously occupies the service point while other queues barely move. Practical designs therefore often limit priority traffic and reserve resources for other classes.

Shaping and policing: two officers with different personalities

Two new officers join the Packet Highway.

The first is patient. When too many cars arrive, he sends them to a parking area and releases them at a controlled pace. This is traffic shaping: regulating the sending rate by buffering and scheduling traffic. It can add delay, but it smooths bursts.

The second officer has no parking lot.

— Limit exceeded!
— Can I wait?
— No.
— What now?
— Turn around. Next!

That's traffic policing: enforcing a rate or traffic contract by dropping packets that exceed it or remarking them, depending on the policy. Policing is not simply another name for shaping.

— Which officer is better?
— Depends on the goal.
— As usual.
— Welcome to computer networking.

The giant parking lot returns: FQ-CoDel and CAKE

After the bufferbloat episode, the manager finally understands that an enormous parking lot before the booth isn't the answer to everything.

— What if we build an EVEN BIGGER parking lot?
— No.
— With a separate floor for cats?
— Still no.

FQ-CoDel combines per-flow fairness with active queue management. It helps stop a single large transfer from monopolising the line and reduces excessive waiting.

CAKE combines techniques including shaping, flow fairness and delay management; some configurations also take traffic classes into account.

Crucially, fair queueing isn't the same as giving one application unlimited priority. Often, keeping one large transfer from blocking every small packet is enough to transform the experience.

— So the ambulance gets through faster?
— More likely when things are congested.
— And the cat truck?
— It still gets there.
— Social justice on wheels!

Where should the traffic officer stand?

Here's a puzzle. The Internet uplink can send at 100 Mb/s, while the local router has a 1 Gb/s port.

— We've enabled QoS on the router!
— Where does congestion actually occur?
— On the 100 Mb/s link.
— How fast does your router send towards it?
— At a gigabit.
— So you're passing the traffic jam along before managing it.

Effective queue management requires influence over the actual bottleneck. For outbound traffic, shaping is often set slightly below the real available link rate so queues form on the device we control, rather than somewhere upstream.

Inbound traffic is harder: packets may already have saturated the provider's link before reaching our router. We can influence some flows, but we can't undo congestion that happened before packets arrived.

— So our officer should stand behind the traffic jam and shout “Don't enter”?
— He can try. The drivers will hear him too late.

Does QoS work across the entire Internet?

The manager proudly attaches an “URGENT” sticker to a packet.

— Now it has priority all the way to Australia!
— Not necessarily.
— But it has a sticker!
— Did the Australian provider agree to honour your sticker?
— No.
— Exactly.

DSCP markings may be preserved, modified or ignored as packets cross different networks. The public Internet doesn't guarantee that a priority assigned at one end will be honoured along the entire path.

QoS is most predictable where we control the devices and policy, or where providers have agreed on service treatment. With tunnels, we may also need to consider how markings are propagated into the outer packet header.

It's like emergency vehicle regulations: official rules apply within an organised system, but sticking a sign on your own windscreen doesn't magically turn your car into an ambulance.

QoS can't repair a broken bridge

A new support ticket arrives.

— We're losing packets!
— On which link?
— The damaged one.
— What did you do?
— We marked everything highest priority!

QoS cannot fix damaged cables, incorrect routes, misconfiguration, radio interference or capacity that simply isn't available. It can divide existing resources more sensibly, but it cannot create them.

If every packet receives the highest priority, none of them is particularly privileged.

— Boss, every car is now an ambulance!
— Excellent!
— And every ambulance is stuck in traffic.
— Perhaps we should give them sirens?

Finale: cats versus the ambulance

After a week of traffic management on the Packet Highway, things look much better. Voice calls no longer break up whenever a large file transfer starts, SSH responds more promptly, and overnight backups still reach their destination.

A familiar truck pulls up at the booth.

— Hello, I've got 4K cat videos.
— Regular lane, please.
— But it's urgent!
— Why?
— The cat just jumped off the sofa and the entire Internet needs to see it!
— I understand. Regular lane.

Moments later, an ambulance arrives. The officer lets it through immediately.

The manager looks puzzled.

— So are they equal or not?
— They have an equal right to use the road. Not every delivery has the same deadline.
— What about the cat?
— The cat can wait.
— You clearly don't know cats.

🤣

A new notice appears on the Packet Highway: “Emergency traffic is given priority. Drivers claiming to carry critical materials may be asked to show the contents of their trailers. A cat video does not constitute evidence of emergency vehicle status.”