Dynamic Routing with GCP’s Network Connectivity Center

As a cloud network operator on GCP, you might routinely invest untold hours maintaining virtual machines in the above diagram to function as a network router or firewall appliance on Google Cloud. Network Connectivity Center, Google Cloud’s hub for managing enterprise networks, can give you those hours back.

The Trouble with Static Route Networks on GCP

Here’s a typical workflow pattern; network admins install third-party virtual router appliances on Google Cloud to manage their VPC network the same way they would on-premises. In the past, GCP cloud router do not support dynamic routing with a virtual network appliance instance and network operators are stuck managing a valid list of static routes.

The diagram illustrates the inherited complexity of operating a network configured with static routes on Google Cloud from the previous diagram.

For every region created in both VPC A and B, static routes are configured to take the virtual router appliance’s vNIC IP as the next hop address to reach the destination network. The regional router appliance VM also needs to be configured with static routing entries.

Given, Sn = N (N — 1) / 2

Where S = number of remote spoke network(s)

N = number of static route configuration(s)

Scaling from three full mesh spokes to six spokes increases routing configuration complexity up to five times.

A single configuration mistake can cost hours of troubleshooting. As the demand for network connectivity grows on GCP to meet business scale, this quickly becomes a network operator nightmare.

What are some current workaround(s)?

A single Global VPC. GCP network operators simply allocate subnet ranges per region within a VPC and Google’s global VPC routes inter-region traffic by default. This single shared VPC design pattern on Google Cloud may be enough to meet your business requirements if you do not need a modular VPC design.

If a single VPC is not enough to micro segment the network architecture consider using VPC peering. On GCP, network operators can automatically import/export routes between two different VPCs. This is implemented by a simple on/off API configuration and NOT with a routing protocol/

Both workarounds avoid the use of a virtual router appliance, potentially violating the enterprise requirement for using advance routing and security features only available on those appliances.

Network Connectivity Center as the solution

Dynamic routing protocols such as BGP were invented to mitigate the toil of maintaining an up-to-date routing table. In fact, the entire internet is built using BGP.

The diagram below shows a solution based on Network Connectivity Center where regional cloud routers in both VPC A and VPC B can now dynamically exchange BGP updates with a virtual network appliance.

This allows routes to be dynamically exchanged once a cloud router is configured to peer BGP with a virtual interface. The solution is enabled by configuring the virtual router appliance virtual interface as a Network Connectivity Center spoke and attaching the spoke to a Network Connectivity Center hub.

Create the NCC hub

gcloud network-connectivity hubs create ncc-hub

Configure VM appliance’s vNIC0 as a spoke.

gcloud network-connectivity spokes linked-router-appliances create spoke1 \
--hub=ncc-hub \
--router-appliance=instance="https://www.googleapis.com/compute/projects/$projectname/zones/us-central1-a/instances/vm-r1",ip=10.1.1.3 \
--region=us-central1 \
--site-to-site-data-transfer

The NCC spoke configuration above unlocks the Cloud Router for BGP peering with a Google Compute Engine that is configured as virtual router appliance.

Configure a Cloud Router in VPC A

gcloud compute routers create cloud-router-a \
--region=us-central1 \
--network=vpc-a \
--asn=65002 \
--set-advertisement-groups=all_subnets \
--advertisement-mode=custom

Configure interface0 and interface1 on the cloud router to exchange BGP messages.

gcloud compute routers add-interface cloud-router-a \
--region=us-central1 \
--subnetwork=workload-subnet1 \
--interface-name=int0 \
--ip-address=10.1.1.10

gcloud compute routers add-interface cloud-router-a \
--region=us-central1 \
--subnetwork=workload-subnet1 \
--interface-name=int1 \
--ip-address=10.1.1.11 \
--redundant-interface=int0

Verify the BGP state on the cloud router

gcloud compute routers get-status cloud-router-a --region=us-central1

As your business expands to different regions, simply configure the new virtual router appliance as a spoke to the existing hub.

What’s happening under the hood with NCC?

1. The Network Connectivity Center hub serves as a central control routing plane for a group of spokes. A Network Connectivity Center spoke is a collection of VPN tunnels, Interconnects, or virtual router appliances that advertise the same on-prem prefixes into Google Cloud. A Network Connectivity Center hub-and-spoke configuration orchestrates how Google cloud routers advertise prefixes using BGP.
2. NCC instructs Cloud Router to propagate route advertisements to other VPN tunnels, interconnects, and virtual router appliances that belong to other spokes attached to the same Network Connectivity Center hub.
3. By default, Cloud Router supports BGP peering over a dedicated interconnect and Cloud VPN with on-prem equipment using link local address 169.254.x.x/16. BGP peering for dynamic route exchange with a virtual router appliance does not work. This leaves network operators the task of building complex static route configurations on Google Cloud and virtual router appliances to ensure data path connectivity.
4. Configuring the router appliance’s vNic as a Network Connectivity Center spoke enables BGP peering with the Cloud Router’s virtual interface. Google Cloud Router won’t establish BGP to the virtual router appliance without its vNIC being a Network Connectivity Center spoke.

This diagram describes Network Connectivity Center data path setup and route message exchange.

1. On prem CPE and VM router appliance establishes a BGP session
2. Google Cloud network operators configure the router appliance’s vNIC0 as Network Connectivity Center spoke 1.
3. Cloud Router’s interface 0 is configured to establish a BGP neighbor relationship on the 10.1.1.0/24 network. The Network Connectivity Center hub converges all routes learned from router appliance into the VPC routing table.

Recap

Network Connectivity Center does NOT manage the data plane components responsible for routing data packets — rather, it aggregates control plane BGP routing information from VPN tunnels, interconnects, and virtual router appliances that are configured as Network Connectivity Center spokes.

When something goes wrong, operators don’t have to waste time troubleshooting Network Connectivity Center configurations, but rather spend their time to check the network elements along the data path and router appliance configuration.

GCP’s network intelligence center’s connectivity test can help visualize all the GCP components that touches the data packet.