MTU and MSS: when the truck cannot fit under the bridge

Something extraordinary has happened on the Packet Highway. The transport company manager has bought a new truck. It's so enormous that the bus parked next to it looks like a delivery van. Accounting fainted at the invoice, while marketing immediately announced “A revolution in next-generation logistics”.

— Gentlemen! No more tiny vehicles!
— Why, boss?
— One enormous truck carries as much as ten little ones!
— But it won't fit everywhere.
— Nonsense! All the roads are connected! LOAD IT!

They loaded it. The truck passed the warehouse, two junctions and three toll booths before stopping at a low bridge. A height-limit sign hung above the road.

— Boss, the truck won't fit.
— Try the other lane!
— Same bridge.
— Drive faster, then!
— BOSS! IT'S TOO BIG!
— Have you tried turning the engine off and on again?
— ...
— Hello? Can you hear me?
— Unfortunately, perfectly.

🤣

MTU: how much can pass at once?

Computer networks have low bridges too. One link can carry a bigger packet; another has a smaller limit. MTU (Maximum Transmission Unit) is the maximum size of a network-layer packet that a particular link can carry without having to split it.

A typical Ethernet network has an MTU of 1500 bytes. This is the size of the IP packet, including its IP header, not the Ethernet frame header and trailer. Let's not confuse the truck with the entire toll booth.

— I've got 1501 bytes. That's only one extra!
— The limit is 1500.
— Can't you make an exception?
— Sir, this is a network device, not airport hand luggage control.

The key point: the tightest section of the complete route sets the limit. A magnificent wide highway is no help when there's a low bridge two towns later.

Why not split the cargo? IPv4 fragmentation

A warehouse worker looks at the bridge and has an idea.

— What if we sent the cargo in several smaller vehicles?
— SEVERAL?! We just bought this enormous truck!
— Smaller ones would fit.
— Then why did we buy the enormous one?
— That's a question for Purchasing.

IPv4 supports fragmentation: an oversized packet may be split into smaller fragments. Each travels independently, and the destination tries to reassemble the original packet. But splitting has costs: extra overhead, and losing one fragment may prevent the whole original packet from being recovered.

An IPv4 router may fragment a packet only when fragmentation is permitted. If the packet has the DF (Don't Fragment) flag set, it must not be split in transit.

— Too big. Split your load.
— Can't. My paperwork says “DO NOT SPLIT”.
— Who issued that?
— The sender.
— THEN TELL THE SENDER TO SEND A SMALLER TRUCK!

The router discards the packet and should send an ICMP Destination Unreachable — Fragmentation Needed message, ideally indicating the supported MTU.

IPv6: back to the warehouse, please

IPv6 is stricter: routers along the path do not fragment packets. When a packet is too large for the next link, the router discards it and sends ICMPv6 Packet Too Big. The sender must adjust the size of later packets. If IPv6 fragmentation is needed, it is performed by the source device.

— Can you unload the truck here on the hard shoulder?
— No. With IPv6, that's the sender's job.
— But the warehouse is 800 kilometres away!
— You should have checked the route.
— Do you know how expensive this delivery is?!
— No. I look after the bridge.

IPv6 requires links to support an MTU of at least 1280 bytes. That does not mean every complete route has exactly that limit.

Path MTU Discovery: find the lowest bridge

The manager hires a route inspector.

— First bridge: 1500. Second: 1492. Third: 1400.
— So which truck do we send?
— One that fits under the lowest: 1400.
— CAN'T WE JUST RAISE THE BRIDGE?!
— If it's ours, perhaps.
— And if it belongs to another provider?
— We could write and say our Internet doesn't fit.

🤣

Path MTU Discovery (PMTUD) is used to discover the maximum packet size that can traverse the whole path. The classic approach uses ICMP feedback: fragmentation-needed messages in IPv4 and packet-too-big messages in IPv6. There are also approaches that probe packet sizes at the transport layer, such as PLPMTUD.

If messages about oversized packets are carelessly blocked, we may get a PMTU black hole: the sender never learns why larger packets disappear.

— Boss, ping works!
— So the Internet works!
— But the website won't load.
— HOW?!
— Small cars get through. Big ones vanish.
— Why aren't the big-truck drivers reporting it?
— They are. Their reports aren't getting through either.

🤣🤣

A successful small ping does not prove that large packets can pass. Troubleshooting can involve different probe sizes and observing MTU-related messages, while remembering that not all routers respond to ICMP in the same way.

MSS: cargo is not the whole truck

After the bridge disaster, the manager reviews another invoice.

— Hold on! The truck can carry 1500 units, but you only pack 1460?!
— Correct.
— WHO STOLE THE OTHER FORTY?!
— Nobody. That's room for the shipping paperwork.
— FORTY UNITS OF PAPERWORK?!
— Twenty for IPv4, twenty for TCP.
— In the old days, we delivered cargo, not forms!

And that's why MTU and MSS are different. MTU describes the whole IP packet. MSS (Maximum Segment Size) is the maximum TCP data payload in a single segment, excluding the IP and TCP headers.

For typical IPv4 with MTU 1500, a 20-byte IPv4 header and a basic 20-byte TCP header, the result is MSS 1460 bytes. With IPv6, whose base header takes 40 bytes, the comparable value is usually 1440 bytes. These are simple examples without extra IP headers; real connections may differ.

During connection setup, TCP can announce an MSS value in SYN packets. Each endpoint advertises the amount of TCP data it is prepared to receive. This is not a guarantee that every bridge on the route has enough clearance.

— Does UDP negotiate MSS as well?
— No. MSS is a TCP concept.
— So how does UDP know how much to send?
— It needs to account for its own overhead and the path's limits.
— So we have to think again?
— Unfortunately, boss.

MSS clamping: tell the warehouse before departure

The administrator decides that enough trucks have been sent out blindly. When TCP endpoints negotiate their maximum payload sizes, a device along the path can reduce the advertised MSS to a suitable value. This is called MSS clamping.

— From now on, pack no more than 1360 units!
— But 1460 fitted before!
— Before, we weren't going through a tunnel.
— Why does the tunnel leave less room?
— Because we're carrying one truck inside another truck.
— WHAT?!
— VPN.
— BLOODY HELL, WHO THOUGHT OF THAT?!
— The security department.

🤣🤣

A tunnel, VPN or PPPoE can add headers and reduce the space available for the inner packet. There is no single MSS value that works for every tunnel; the correct value depends on the real link limits and encapsulation overhead.

MSS clamping affects TCP. It doesn't automatically fix oversized UDP datagrams or all IP traffic, and it is no replacement for correct ICMP handling, PMTUD and MTU configuration.

What should we check when trucks pile up under the bridge?

If small requests work but bigger transfers hang, especially over VPN or PPPoE, check MTUs along the path, encapsulation overhead, MTU-related ICMP messages, PMTUD behaviour and the MSS negotiated by TCP. “Ping replies” is not enough of a diagnosis.

Of course, not every website that refuses to load has an MTU problem. But if packets above a certain size disappear, this suspect deserves questioning.

— Boss, we found the cause!
— What was broken?
— Nothing. We were sending packets that were too large.
— And the bridge?
— Still there.
— So we could have bought a smaller truck?
— Yes.
— Please don't tell Accounting.

🤣

The Packet Highway Authority reminds drivers that maximum-height signs also apply to vehicles marked “VERY IMPORTANT”. The manager has requested that MTU be renamed “suggested packet size”. Request denied.