Understanding GRE and IPsec: Transport Mode vs Tunnel Mode (Lab Analysis)

When studying VPN technologies such as GRE(Generic Routing Encapsulation) and IPsec, the theory describes packet encapsulation but does not clearly show what actually happens inside a network.

To understand this behaviour, I built a small lab using Cisco (CML) and analysed packet captures to observe how packets change when GRE and IPsec are applied.

The goal was to answer three questions:

1. What does GRE encapsulation look like in a packet capture?
2. What changes when IPsec transport mode is applied?
3. How does IPsec tunnel mode differ from transport mode?

Lab Topology

The lab consisted of two enterprise routers connected across an intermediate network simulating the internet.

LAN A → Router A → Internet → Router B → LAN B

The routers were configured with:

• A GRE tunnel between the two routers
• Routing across the GRE tunnel
• IPsec encryption applied to the tunnel traffic

Packet captures were taken on the internet-facing interfaces to observe the encapsulated traffic.

Step 1 — Observing GRE Encapsulation

After establishing the GRE tunnel, traffic between hosts on both sides of the network was captured.

Wireshark revealed the following packet structure:

Outer IP Header (Router A → Router B) GRE Header Inner IP Packet (Host A → Host B) ICMP Payload

This confirmed that GRE encapsulates the entire original packet inside a new IP header.

The outer header allows the packet to traverse intermediate networks, while the inner packet remains intact.

Step 2 — GRE with IPsec Transport Mode

Next, IPsec was configured using transport mode.

In transport mode, IPsec encrypts the payload but leaves the outer IP header unchanged.

Packet capture now showed:

Outer IP Header ESP Header Encrypted Payload

The GRE header and inner IP packet were now hidden inside the encrypted ESP payload.

This demonstrated that transport mode protects the GRE tunnel traffic while still using the original outer IP header.

Step 3 — GRE with IPsec Tunnel Mode

The configuration was then modified to use IPsec tunnel mode.

Tunnel mode encapsulates the entire packet before encryption.

The structure becomes:

Outer IP Header (IPsec peer addresses) ESP Header Encrypted Payload

Inside the encrypted payload is:

Inner IP Header (GRE endpoints) GRE Header Original IP Packet ICMP Payload

This introduces an additional IP header compared to transport mode, increasing packet size and overhead.

Key Differences Between Transport and Tunnel Mode

Transport Mode

• Encrypts the payload of the packet
• Original IP header remains visible
• Lower overhead
• Commonly used with GRE tunnels

Tunnel Mode

• Encrypts the entire packet
• Adds a new outer IP header
• Higher overhead
• Often used for site-to-site VPNs without GRE

Why GRE and IPsec Are Often Combined

IPsec alone encrypts traffic but does not natively support certain routing protocols(OSPF, EIGRP) and multicast traffic.

GRE solves this by encapsulating the original traffic before encryption.

The typical production design is:

GRE over IPsec (transport mode)

This combination provides:

• Routing protocol support
• Multicast capability
• Secure encrypted transport

Observations from Packet Analysis

Using packet captures helped visualise several key networking concepts:

• GRE adds a new IP header and GRE header
• IPsec encrypts the GRE packet inside ESP
• Tunnel mode introduces additional overhead
• Transport mode is more efficient when GRE already provides the tunnel

Conclusion

Building the lab and analysing packet captures provided a much clearer understanding of how tunnelling and encryption interact.

While the theoretical differences between GRE and IPsec are often explained conceptually, observing real packets in Wireshark reveals how these protocols actually modify traffic..

For anyone studying networking, combining lab work with packet analysis is an effective ways to build intuition about how protocols behave in real networks.