PrivateLink vs PNI: Secure Kafka Connectivity on AWS with Confluent Cloud (Platform Architecture Guide)

This article is a short interlude in my two-part series on architecting secure Kafka connectivity on AWS with Confluent Cloud.

In **Part I**, I showcased how to use Terraform to stand up a fully private, production-grade connectivity architecture between AWS and Confluent Cloud using AWS PrivateLink and Cluster Linking.

This interlude is about something just as important as networking patterns:

Separation of Concerns in Infrastructure-as-Code (IaC).

Separation of Concerns

Revisiting the original architecture and GitHub repo, I realized something important.

Although the repo works and correctly demonstrates how to configure PrivateLink connectivity with two VPCs connected to Confluent Cloud Enterprise clusters, it is not how you would structure this in a real production environment.

Originally, the Terraform configuration contained:

• AWS PrivateLink networking infrastructure
• Confluent Cloud application resources
• Cluster Linking configuration
• Service accounts and API keys

All inside a single Terraform workspace.

That’s fine for a demo.

It’s not fine for a production platform engineering team.

Demo mindset vs production discipline

In reality, you would separate these into two independent Terraform workspaces with separate GitHub repos:

1. AWS PrivateLink Network & Infrastructure
2. Confluent Cloud Application Resources

Why?

There are several reasons to separate these concerns into different Terraform workspaces:

• Different Lifecycles: The AWS PrivateLink Network and infrastructure are set up once and used for multiple applications that connect to Confluent Cloud. Confluent Cloud Application Resources are specific to each application you want to connect to Confluent Cloud and may have different lifecycles and requirements. By keeping them in separate Terraform workspaces, you can manage their lifecycles independently and avoid conflicts or issues that can arise from sharing the same workspace.
• Different Blast Radius: Keeping them in separate Terraform workspaces lets you manage them independently and avoid conflicts or issues that can arise from sharing the same workspace.
• Different Access Controls: By keeping them in separate Terraform workspaces, you can manage access controls independently and ensure that only the appropriate teams or individuals have access to the resources that they need to manage.
• Team Autonomy: In many organizations, the team that manages the AWS PrivateLink Network and Infrastructure may be different from the team that manages the Confluent Cloud Application Resources. That is, the platform engineering team manages the infrastructure, while the application/data teams manage the Kafka resources for their own applications. By keeping them in separate Terraform workspaces, you can allow each team to manage their own resources without worrying about conflicts or issues with the other team’s resources.
• Dependency Ordering without Coupling: By keeping them in separate Terraform workspaces, you can manage dependencies between resources without coupling them in the same workspace. This allows the platform engineering and networking teams to manage the AWS PrivateLink Network & Infrastructure, while application/data teams manage their Confluent Cloud resources without cross-team friction. For example, the platform engineering team can switch from AWS PrivateLink to AWS PNI without worrying about how it will impact the application/data teams’ resources, and the application/data teams can add additional service accounts and other Kafka resources without worrying about how it will impact the platform engineering team’s resources.

Reimagining the Example

So I refactored the original implementation into two independent GitHub repos, each mapped to its own Terraform workspace.

Repo 1: AWS PrivateLink Network & Infrastructure

The [**iac-cc-aws_privatelink-infrastructure_networking-example](https://github.com/j3-signalroom/iac-cc-aws_privatelink-infrastructure_networking-example)** repo provisions:

• AWS VPC networking
• PrivateLink endpoints and attachments to Confluent Cloud
• Centralized Route 53 Private Hosted Zone (PHZ) for all PrivateLink attachments
• DNS strategy that accounts for Confluent’s shared PrivateLink namespace constraint
• Two Enterprise-tier Kafka clusters (Sandbox + Shared)

The architectural nuance here is subtle but critical:

**Confluent PrivateLink attachments share a non-unique DNS namespace, while AWS Route 53 prevents associating multiple Private Hosted Zones (PHZs) with the same domain name across overlapping VPC associations. As a result, separate PHZs cannot be created per cluster and distributed across interconnected VPCs**.

Which means you CANNOT create one PHZ per cluster and wire them together.

The solution is a centralized DNS design that cleanly routes traffic across VPCs without namespace collisions.

PrivateLink Architectural Diagram

PrivateLink Terraform Resource Dependency Diagram

Repo 2: Confluent Cloud Application Resources

The [**iac-cc-app_resources-example](https://github.com/j3-signalroom/iac-cc-app_resources-example)** repo provisions:

• Bidirectional Cluster Linking
• Mirror topic
• Service accounts and API keys
• Automated credential rotation
• Secure storage of credentials in AWS Secrets Manager

All of it is isolated from networking & infrastructure concerns.

Application logic.

No network state.

Clean separation of concerns.

Confluent Cloud Application Resources Architectural Diagram

Confluent Cloud Application Resources Dependency Diagram

The Bigger Architectural Lesson

This interlude isn’t really about PrivateLink vs PNI, or even about Kafka connectivity patterns on AWS.

It’s about platform maturity.

Infrastructure-as-Code (IaC) is not just about automating resource provisioning.

It encodes:

• Ownership boundaries
• Team responsibilities
• Organizational structure
• Security posture
• Operational risk tolerance

If everything lives in one Terraform workspace, your architecture may work…

…but your operating model will not scale.

Separating foundational infrastructure from application-layer Kafka resources is not over-engineering.

It’s a production discipline.

What’s Next

In Part II, we move beyond AWS PrivateLink and into Confluent Private Network Interface (PNI), a very different approach to private connectivity.

PrivateLink is familiar. PNI is architectural by design.

Instead of stitching together interface endpoints and DNS workarounds, PNI extends Confluent Cloud directly into your VPC using private IP space, changing the networking model entirely.

In Part II, we’ll explore:

• How PNI fundamentally differs from PrivateLink
• What it changes about routing, DNS, and operational ownership
• Where it simplifies architecture — and where it introduces trade-offs
• When platform teams should seriously consider migrating

Because once you separate infrastructure from application concerns, the next strategic question becomes:

Are you optimizing around workaround patterns… or around architectural alignment?

Part II dives into that decision and its implications for secure, production-grade Kafka connectivity on AWS.

Thank you for reading. 😊

If you have any questions or comments, please leave them below.

— J3

If you liked the post, please clap and follow me on Medium and LinkedIn!