Finding the Sweet Spot…In regard to MTU Size

Imagine your data packets as carefully dressed travelers, each in a perfectly tailored suit. Now, imagine they have to walk through a series of doorways to reach their destination. Most doorways are standard size, but some are unexpectedly narrow, and some even have an extra security turnstile. If your traveler’s suit is too big for any doorway, bad things happen: either they get chopped up (fragmented), or they get sent back, or worse, they just disappear into the ether.

Welcome to the world of Maximum Transmission Unit (MTU), a seemingly innocuous setting that can cause untold grief if misconfigured. Finding the “sweet spot” for your MTU, especially when dealing with network tunnels, is less about intuition and more about methodical detective work.

MTU: Your Packet’s Suit Size (and Why It’s a Problem Child)

At its core, MTU is the largest size packet (including headers) that a network interface can send without needing to be chopped up. For standard Ethernet networks, this is almost always 1500 bytes.

When a packet larger than the MTU of any link in its path tries to traverse that link, one of two things happens:

1. Fragmentation: The packet is split into smaller pieces that do fit. Each piece then gets its own set of headers.

• Why it’s bad: It’s inefficient. More packets mean more processing overhead for routers and end devices. It increases latency and can consume more CPU cycles, turning your smooth data flow into a choppy mess. Imagine chopping an essay into single words, putting each word in its own envelope, and mailing them. Inefficient!

2. Dropping: If the packet has a special “Don’t Fragment” (DF) bit set (more on that in a moment), it will be dropped rather than fragmented. This is worse for connectivity, but better for diagnostics.

Most network engineers agree: avoid fragmentation at all costs.

The DF Bit: Your Packet’s “No Fragmentation Allowed!” Sticker

This is where our detective work begins. Every IP packet has a set of flags in its header, and one of them is the Don’t Fragment (DF) bit.

When this bit is set, it’s like a tiny, authoritative voice on the packet screaming, “I must arrive whole, or not at all!” If a router in the path receives this packet and finds it’s too big for the next hop’s MTU, instead of fragmenting it, the router simply drops the packet and typically sends back an ICMP “Fragmentation Needed and DF set” (Type 3, Code 4) message to the source.

This ICMP message is our golden ticket! It tells us exactly where the packet got stuck and what size MTU it needed. This process is the foundation of Path MTU Discovery (PMTUD), a mechanism hosts use to dynamically figure out the largest MTU they can use to a destination without fragmentation.

Finding the Path MTU: The Ping Test Method (The Digital Sizing Chart)

Since we can’t physically shrink doorways for our data packets, we have to find the smallest doorway along the path. This is best done with a series of ping tests from a Linux machine (Windows has similar capabilities, but Linux is often more explicit).

The Linux Ping Command for MTU Discovery:

The key options are -M do (which sets the DF bit to "do not fragment") and -s <packet_size> (which sets the payload size of the ICMP packet).

Important Note: The ping command's -s option specifies the ICMP payload size. The actual IP packet size sent over the wire will be payload_size + 28 bytes (20 bytes for the IP header + 8 bytes for the ICMP header). So, if your target MTU is 1500, your payload should be 1500 - 28 = 1472.

The Process (Trial and Error with Purpose):

1. Start High: Begin with a payload size you suspect might be too large (e.g., 1500 for a regular network, or something like 1400 if you suspect a tunnel).

ping <destination_IP> -M do -s 1500

2. Look for the Error: If the packet is too large, you’ll see output like:

ping: local error: Message too long, mtu=1500 
From <router_ip> icmp_seq=1 Frag needed and DF set (MTU 1400)

This message is your clue! It tells you the maximum MTU allowed by the router that dropped your packet (in this example, 1400).

3. Decrease and Retest: Reduce your -s (payload) value based on the MTU reported in the error, or by a smaller increment, and try again.

• If the error says MTU 1400, then your new -s should be 1400 - 28 = 1372.
• If you just get “Message too long” without a specific MTU, try dropping by 10 or 20 bytes at a time.

ping <destination_IP> -M do -s 1372 # Trying the new size

4. Find the Sweet Spot: Keep reducing the payload size until the ping command starts receiving replies without the "Frag needed and DF set" error. The moment it starts working, take that payload size, add 28 back to it, and that's your Path MTU!

• Congratulations! You’ve just digitally “tried on” different suit sizes for your packets until you found one that fits through every single tiny doorway on the network. It’s tedious, but oh-so-satisfying.

Troubleshooting MTU Mismatches: The Network Engineer’s Headache (and How to Cure It)

MTU problems are notoriously frustrating because they often manifest in unpredictable ways. Your basic pings work, web Browse seems okay, but then that critical application hangs, or your VPN connects but passes no traffic. This is the hallmark of an MTU mismatch.

Common Symptoms of MTU Mismatches:

• Sporadic Connectivity: Small packets (like DNS queries, small web page elements, basic pings without -s) work fine, but larger data transfers (file downloads, video streaming, VPN tunnels, RDP sessions, FTP data channels) fail, stall, or are excruciatingly slow.
• VPN Tunnel Issues: The VPN establishes, but you can’t reach resources on the other side, or only very small packets get through. This is a classic symptom of a tunnel MTU issue.
• Application-Specific Failures: Some applications (especially older ones or those sensitive to fragmentation) might simply stop working or time out without a clear error.
• Mysterious Connection Drops: TCP sessions might randomly reset or hang.

Troubleshooting Steps (Beyond the Ping DF Test):

1. Verify MTU on All Interfaces in the Path: This is your starting point. You need to confirm the MTU setting on every single network interface between your source and destination. A single misconfigured hop can ruin your day.

• Linux:

ip link show

• Look for mtu <number> in the output for each interface.
• Windows (Command Prompt/PowerShell as Admin):

netsh interface ipv4 show subinterfaces

• This will list the MTU for all your network adapters.
• Network Devices (Routers/Switches/Firewalls): Log into each device in the path and check its interface configurations. Commands vary by vendor (e.g., show interfaces <interface_name> on Cisco, show interfaces <interface_name> | match mtu on Juniper, or web UI for consumer routers).

2. Firewall Considerations (The Silent Killer of PMTUD): A very common culprit for “black hole” connections (where traffic goes out but no reply comes back) is a firewall blocking ICMP “Fragmentation Needed and DF set” messages (ICMP Type 3, Code 4). If the router tries to send this message back to your host, but a firewall blocks it, your host never learns that its packets are too big. It keeps sending large packets that are dropped, leading to timeouts.

• Check Firewall Rules: Review firewall rules on all devices in the path to ensure ICMP Type 3, Code 4 messages are allowed, especially on WAN edges and VPN endpoints.

3. Packet Capture (Your Best Friend: Wireshark/tcpdump): This is the ultimate diagnostic tool. Run a packet capture on both the sending and receiving devices, and ideally, on any intermediate routers/firewalls if possible.

What to Look For:

• Packets Larger Than MTU: Are packets being sent out an interface that are actually larger than its configured MTU? This indicates fragmentation or a potential misconfiguration.
• Fragmentation: If DF is not set, are you seeing packets being fragmented (More fragments bit set, same ID, different offsets)? Even if it works, it's inefficient.
• Missing ICMP Type 3 Code 4: If you’re sending packets with DF set and they’re not getting through, but you’re not seeing the “Frag needed” ICMP message return, that’s a strong sign a firewall is blocking it.
• Retransmissions/Excessive Drops: For TCP, a high number of retransmissions (seen in iPerf3 or Wireshark) might indicate packet loss due to MTU issues. For UDP, just plain drops.
• TCP Session Failures: Observe the TCP handshake. Does it complete? Do the first few small data packets go through, and then the connection stalls when larger data segments are sent? This is typical.

4. Router/Switch Logs: Check the logs on your network devices for any messages related to MTU, fragmentation, or packet drops on interfaces.

5. TCP MSS Clamping (adjust-mss): While adjust-mss is a fix to prevent MTU issues, its absence or misconfiguration can be a root cause. If your devices (especially tunnel endpoints) aren't correctly clamping MSS, that's a problem. Verify it's enabled and configured with the correct value (Path MTU - 40 bytes) on tunnel interfaces or WAN interfaces where necessary.

The Debugging Mindset: Troubleshooting MTU is about being methodical. Start from the source, trace the path hop by hop, and use your tools (ping DF, ip link show, netsh, packet captures) to identify where the packet is being dropped or fragmented. It's like finding the one slightly-too-small shoe in a very long obstacle course.

Tunnels: The MTU’s Bermuda Triangle

This is where MTU discovery becomes even more critical. Network tunnels (like VPNs, GRE, IPSec, etc.) work by encapsulating your original packets inside another, outer packet.

• Original Packet: Your data, plus its standard IP and TCP/UDP headers.
• Encapsulation: The tunnel adds another set of headers (e.g., GRE header, new IP header, IPSec headers) around your original packet.

The Problem: The entire, newly encapsulated packet now has to fit through the physical MTU of the tunnel endpoint’s interface. If your physical interface is 1500 bytes, and your tunnel adds, say, 60 bytes of overhead, then the effective MTU for the data inside the tunnel is now 1500 - 60 = 1440 bytes.

If a device sends a 1500-byte packet into this tunnel, it will hit the tunnel endpoint, get encapsulated into a 1560-byte packet, and then attempt to exit the physical interface (which only allows 1500 bytes). Result? Fragmentation, or if the DF bit is set, a dropped packet inside the tunnel, leading to mysteriously broken connections. This is why VPNs and other tunnels often feel like they’re in their own MTU Bermuda Triangle, where packets go to disappear.

MSS: TCP’s Internal Packet Agreement

While MTU is about the maximum size of the entire IP packet, MSS (Maximum Segment Size) is TCP’s best friend. MSS specifies the largest amount of data (the payload, without IP or TCP headers) that a TCP sender will put into a single segment.

• Relationship: MSS = MTU - (IP Header Size + TCP Header Size). Since standard IP and TCP headers are 20 bytes each, MSS = MTU - 40 bytes. So, for a standard 1500 MTU, the MSS is 1460 bytes.

During the TCP handshake (the initial SYN/SYN-ACK packets), both ends of a TCP connection advertise their MSS. They then agree to use the lower of the two advertised MSS values. This ensures that TCP doesn’t try to send segments that are too big for the direct link between the two endpoints.

Auto-MSS & Adjust-MSS: The Network’s Tailors

Ideally, PMTUD would handle everything, telling devices to send smaller packets if a link has a lower MTU. But PMTUD can fail (e.g., firewalls blocking ICMP “Frag Needed” messages, as firewalls are wont to do). This is where network devices step in as tailors for your packets.

• Auto-MSS / TCP MSS Clamping: Many modern routers and firewalls have a feature to automatically adjust (or “clamp”) the MSS value advertised in TCP SYN/SYN-ACK packets.
• How it works for Tunnels: When a TCP SYN packet enters a router that’s about to send it into a tunnel, the router looks at the MSS value advertised by the client. It then calculates: Physical_Interface_MTU - Tunnel_Overhead - IP_Header - TCP_Header. If this calculated value is smaller than the client's advertised MSS, the router rewrites the MSS value in the SYN packet to this smaller, safe size.
• Example: Client advertises MSS 1460 (for 1500 MTU). Router sees a tunnel with 60 bytes overhead and a physical MTU of 1500. It calculates the effective MSS for the tunnel as 1500 - 60 (tunnel) - 20 (IP) - 20 (TCP) = 1400. The router then rewrites the client's advertised MSS from 1460 to 1400 in the SYN packet. When the server responds, it will use 1400 as the MSS.
• This ensures that the TCP sender proactively sends segments that are small enough to fit within the tunnel’s effective MTU, preventing fragmentation (or dropping) after encapsulation. It’s the network saying, “Look, your suit is too big for this door; here’s a smaller one that will fit the entire journey.”

The Quest for the Perfect Fit

Finding the MTU sweet spot, especially for tunnels, is crucial for stable and performant network connections. It means meticulous planning, using ping tests with the DF bit to discover the true path MTU, and leveraging adjust-mss (TCP MSS clamping) on your tunnel endpoints to proactively prevent fragmentation for TCP traffic. By mastering these concepts, you'll ensure your data travelers arrive at their destination not only whole, but efficiently and without any wardrobe malfunctions. Go forth and optimize!

Enjoyed this? Then you’ll probably like my book, NSFW Networking: Volume 1. It’s what happens when a network architect gets fed up with vague documentation and decides to write the kind of guide people actually need. It’s blunt, practical, a little unhinged, and surprisingly useful. You can find it here: https://a.co/d/8DAF61z.

More volumes are on the way, including a Safe for Work version for those of you with delicate eyeballs.

Thank you for being a part of the community

Before you go:

• Be sure to clap and follow the writer ️👏️️
• Follow us: **X | [LinkedIn](https://www.linkedin.com/company/inplainenglish/) | [YouTube](https://www.youtube.com/@InPlainEnglish) | [Newsletter](https://newsletter.plainenglish.io/) | [Podcast](https://open.spotify.com/show/7qxylRWKhvZwMz2WuEoua0) | [Twitch](https://twitch.tv/inplainenglish)**
• **Start your own free AI-powered blog on Differ** 🚀
• **Join our content creators community on Discord** 🧑🏻‍💻
• For more content, visit **plainenglish.io + [stackademic.com](https://stackademic.com/)**