MicroNotes XII: Breaking Apart Databases — When Data Becomes the Problem

Notes from Sam Newman’s Microservices Workshop — Bangkok 2025

generated by Gemini

You’ve learned about domain-driven design. You know how to find boundaries. You know what to extract. But here’s where it gets tricky: the data.

When you split services apart, you’re also splitting databases apart. And databases have relationships. Foreign keys. Referential integrity. Joins. All of that stuff that makes relational databases work.

When services are separate, all of that changes. You lose foreign keys, referential integrity, easy joins. You have to find other ways to handle these relationships.

Let’s start with a simple case and work our way up.

Static Reference Data

You’ve got a country codes table. Three-letter ISO codes. Ireland is IRL. Iceland is ISL. Maybe you’ve got currency, capital cities, whatever.

All your services use this table. Catalog service. Finance service. Warehouse service. They all need country codes.

You have several options.

Option 1: Duplicate it.

Each service gets its own copy of the table. Simple. But inconsistency becomes a concern. One service might know about South Sudan and another might not.

The question is whether that inconsistency matters. If the finance service knows about South Sudan because they have customers there, but the warehouse service doesn’t because they don’t have suppliers there, it might not matter.

But if the finance service asks the warehouse service for information about South Sudan, and the warehouse service says “that’s not a real country” then you’ve got a problem.

Sam shared a real example of this. When he moved back to the UK, he rented a house in East London. The house had been split into two flats for over 100 years, but a developer had merged them back together. He contacted the local council to get a rubbish bin. They said his address didn’t exist. But the same council was sending him bills for council tax to that same address they said didn’t exist.

One part of the council used one computer system for waste management. Another part used a different system for taxation. They weren’t updating their address files at the same frequency. The waste management system took months to update. The taxation system was kept up to date because they need to tax people.

So you can have inconsistency in your system. The question is whether you surface that inconsistency to your users. You can decide which version of the data wins. You don’t have to show users different versions of the same data on the same screen.

Inconsistency isn’t always a problem. But when services need to communicate and expect to see consistent information, it becomes a problem.

Option 2: Dedicated service.

Create a country codes service. All services call it. This avoids inconsistency. But now you’ve added a dependency. Someone has to own this service. Someone has to make sure it’s running.

The question is whether it’s worth it.

If the cost of creating and managing a service is high, it needs to provide a lot of value to justify it. If the cost is low, it becomes more attractive. It depends on your environment.

This data is really easy to cache. The volatility is extremely low. You can cache it for about five years and it would be fine. The Czech Republic changes its name more frequently than that.

In fact, why even have a database? There are maybe 300 rows. Why not just have the service hard-code it? Then everything is served straight out of memory. It will be faster than putting the data into a database.

Option 3: Shared library.

For a small amount of data, you could include it as a configuration file or a library. Each service includes the same JSON file or the same library.

You’ve removed the network dependency. You can monitor version drift. It’s not a bad idea for small, static data.

When you share behavior through a library in a microservices architecture, you have to be open to the fact that when you roll out a brand new version of that library, all of the services won’t update at the same time because we have independent deployability. You do get a bit of version drift, but you can monitor that version drift. Quite a few organizations do this with small amounts of static reference data.

Option 4: Shared schema.

Keep it in a dedicated static reference data schema. All services can access it. This is one of the exceptions to the “no shared database” rule. Static reference data is very static.

It rarely changes. Even when it does, you’re usually just adding columns, which is fine.

To decide, start by considering how important it is to have a single view of this data. If you need consistency everywhere, a dedicated service or shared schema makes sense. If you’re okay with some inconsistency, duplication or a shared library might work.

Static reference data is relatively simple. But joining data across services is more complex.

When You Need to Join

Here’s a harder problem. You’ve got a catalog service with product information. You’ve got a finance service with a ledger of sales.

In your monolith, you could do a simple join: get the best sellers from the ledger, join to the catalog to get the names.

But now they’re separate services. You can’t join across services. Even if you technically could, you shouldn’t. That would mean one service reaching into another service’s database.

So you move the join from the database tier to the application tier. You query the finance service for the best-selling item IDs. Then you query the catalog service for the names of those items.

This is probably slower than a database join. The real question is whether it’s still fast enough. If yes, that’s fine. If no, you might need to think about caching or other strategies.

But joins are just one part of the problem.

There’s something else you lose when you split databases apart.

Foreign Keys and Referential Integrity

In your monolith, you had a foreign key relationship.

The ledger table had a column that pointed to the catalog table. The database enforced referential integrity.

If you tried to delete a product that was referenced in the ledger, the database would stop you.

Foreign keys give you three things:

1. Performance

The database creates indices to speed up joins. When you define a foreign key, the database knows you’re going to do joins, so it creates indices to speed up lookups. You can do joins without foreign key relationships, but it gets slower.

One thing to mention: those indices can actually increase the write cost. When you write data, it has to update those indices. Sam spoke to someone who runs the database team for Booking.com.

They actually turn off foreign keys. They have thousands of databases with massive volumes, and for them, the overhead of managing foreign keys is too much. But for most of us, having foreign keys speeds up join operations.

2. Referential integrity

The database enforces that references are valid. You can’t delete something that’s being used. This enforcement goes both ways. Not only when you write a row does the ID have to point to something that exists, but if you try to delete a row that’s being referenced, the database will stop you.

3. Explicitness

Foreign keys make relationships explicit. You can look at the schema and see that this column relates to that table.

This is useful for humans, not just computers. If you didn’t have a foreign key relationship, you’d look at a column with a number in it and wonder what it’s for. You might have to look at the code to see how it’s used. Foreign keys make that relationship explicit in the schema itself.

When you split services apart, you lose all of that. You need to find other ways to handle these relationships.

You have two main challenges: making relationships explicit, and handling deletions.

Making relationships explicit:

You can use something like a pseudo-URI. This idea comes from Phil Calçado, who was an engineering manager at SoundCloud.

Instead of just storing an ID like “123”, you store something like “soundcloud://track/123”. Now it’s clear this is a reference to a track from SoundCloud. You can parse it programmatically. You can see what type of entity it is.

SoundCloud created a library that you can give these IDs to, and it can parse them. If you’re building a REST-based system, you could replace these identifiers with actual permanent URIs to your resources. Some people complain that the word “track” repeats a lot in that column. But disk space is cheap. That’s not the issue you’re dealing with. This is never a disk space issue.

Handling deletions:

This is where it gets tricky.

In your monolith, the database would stop you from deleting something that’s being used.

In microservices, there’s nothing stopping the catalog service from deleting a product, even if the finance service is using it.

You have a few options:

Option 1: Duplicate the data. When you create a ledger entry, copy the product name. Now if the product gets deleted, you still have the name. But if the name changes, do you want to update it?

For a ledger, probably not. A ledger is a record of what happened. When you sold that item, it was called “Death Poker Volume 4.” That’s what you want to keep.

It’s a snapshot in time. Like a photograph. You don’t want photographs to update.

Option 2: Soft delete. Don’t actually delete the product. Mark it as deleted. Filter it out in queries. But keep it in the database so references still work. This is a classic pattern. It’s not hard to implement.

Option 3: Event sourcing. Store state transitions as events instead of current state. Replay events to get current state. This is incredibly elegant and solves interesting problems. But it’s also incredibly hard to implement. It breaks people’s brains. Sam tends to guide people away from it unless they’re really sure they need it.

https://www.linkedin.com/pulse/event-sourcing-pattern-distributed-designpatterns-pratik-pandey/

If you do decide to go down the event sourcing route, start really easy. Do not confuse the events you use for event sourcing with the events you use in communication between services.

They’re different things. If you’re using Kafka to send an event from point A to point B, and you’re still storing the current state of your data in a database table, you’re not event sourcing. Just because you’re using a message broker doesn’t mean you’re doing event sourcing.

If you are interested in using event sourcing, be very careful. Be very sure you need it. And even then, keep all the details hidden inside your service boundary. Consumers of your service should not care if you’re using event sourcing or not.

It’s a pattern where so much of how we think about storing data and working with databases is almost instinctual. Event sourcing fundamentally upends how we think about one of the most important aspects of our application development. It will catch you unaware.

Option 4: Check before deleting. Before the catalog service deletes a product, it asks the finance service if it’s being used. If yes, don’t delete it. If no, delete it.

This sounds reasonable, but it has problems. The catalog service needs to know to ask the finance service. If a new service starts using products, you have to update the deletion code.

And there’s a timing issue. The finance service says “no, not using it.” Then in that millisecond before deletion, someone makes a sale. Now you’ve got inconsistency.

You’re back to the two-phase commit problem. You’d need to lock things. Coordinate. It gets messy.

Option 5: Events. When the catalog service deletes a product, it fires an event. Other services can react to that event however they want.

The finance service might copy the name locally. The recommendation service might delete recommendations for that product.

This is more sophisticated than cascading deletes. Each service decides how to react. But remember: don’t delete items from financial ledgers. That’s how you go to prison. Seriously.

The UK Post Office scandal is a perfect example of why you never mess with financial records. Sam shared the shocking backstory of how the Post Office worked with Fujitsu to roll out a system that tracked financial transactions across its branches — and how everything went disastrously wrong from there.

The system had bugs. It started showing that hundreds of postmasters were stealing money. The Post Office prosecuted them. People went to jail. People lost their homes. Three people committed suicide.

It turned out the system had bugs. Fujitsu and the Post Office knew, and they covered it up. Fujitsu engineers had back doors into the system so they could manually change ledger entries. That completely destroyed the paper trail about what actually happened.

It’s the largest miscarriage of justice in UK legal history. Over a thousand people affected. The average payout per person is going to be around £600,000 in compensation.

Don’t delete items from your ledgers. Don’t mess with financial records.

The consequences are real.

The Real Challenge

Breaking apart databases is hard. There are lots of options.

There’s rarely one obvious right answer. The right answer depends on your context.

Consider your needs: whether you need consistency, whether the data needs to update, whether this is a snapshot in time, what the cost of inconsistency is, and what the cost of coordination is.

Every time you hit a problem, there are usually three or four different ways to solve it. Some are simple. Some are not. Sometimes the business context points you in the right direction.

The key is to think through the options.

Understand the tradeoffs.

Make a decision.

Be ready to change your mind if you learn something new.

What to Remember

So what does this mean for you? Breaking apart databases is one of the hardest parts of migrating to microservices.

But there are patterns and options.

For static reference data, you have choices: duplicate it, create a dedicated service, use a shared library, or use a shared schema. The right choice depends on how important consistency is and what the cost of creating a service is in your environment.

When you need to join data across services, you move the join from the database to the application. It might be slower, but the real question is whether it’s still fast enough.

Foreign keys give you performance, referential integrity, and explicitness. When you lose them, you need to find other ways to get those benefits. You can make relationships explicit with pseudo-URIs. You can handle deletions with soft deletes, data duplication, or events.

The key insight is that there’s rarely one right answer. There are usually multiple options, each with tradeoffs. You need to understand your context: what the business needs, what the costs are, and what the risks are.

Think of it this way: in a monolith, the database enforces a lot of things for you. In microservices, you have to think about those things yourself. You have more control, but also more responsibility.

The database doesn’t solve your problems anymore.

You do. That’s both the challenge and the opportunity.

***All Notes

---

***I: What Are Microservices? II: Forget Service Size — Focus on What Your Team Can Manage III: Microservices Aren’t About Technology — They’re About Team Autonomy IV: Nobody Cares About Your Microservices — Only the Outcome V: Information Hiding — The Discipline That Makes Microservices Work VI: Request-Response vs Event-Driven — Choosing How Services Talk VII: Distributed Transactions Are Sad — Use Sagas Instead VIII: Designing Microservices for the Edge — Lessons from Fish Farming IX: Testing Microservices — Beyond the Test Pyramid X: Migrating to Microservices — Changing the Wheels While the Car Is Moving XI: Domain-Driven Design — Speaking the Same Language XII: Breaking Apart Databases — When Data Becomes the Problem XIII: Resiliency, Observability, and the Reality of Distributed Systems

---