← Back to list

From a single point of failure to a cell-based architecture: How we scaled Mercado Envíos’ stock…

How we migrated Mercado Libre’s second-largest MySQL instance into independent cells using Fury, reducing risk and achieving predictable…

Rafael Silvestri in Tecnología de Mercado Libre · 2025-12-29 21:09 · 38 claps · 7.8 min read
#cell-based-architecture #database-scalability #internaldeveloperplatform #fury
Open on Medium ↗
Wiki topics: 🏛️ · Architecture

From a single point of failure to a cell-based architecture: How we scaled Mercado Envíos’ stock system

How we migrated Mercado Libre’s second-largest MySQL instance into independent cells using Fury, reducing risk and achieving predictable scalability in Fulfillment.

Abstract

What happens when the database supporting a continent’s logistics reaches its limits? We reached that point when a critical regional database, used for inventory operations across Latin America (LATAM), could no longer scale vertically. This article describes how we transitioned from that consolidated model to a cell-based architecture — isolating failures, reducing systemic risk, and improving operational predictability — while keeping logistics running throughout the migration.

When a core system reaches its breaking point

Software architecture uses patterns to prevent local failures from causing global outages. One of them is the cell-based architecture, which is conceptually similar to the naval bulkhead mechanism.

Ships use watertight compartments, or bulkheads, to divide the hull into separate sections. If one compartment floods, the others remain sealed and the ship keeps moving.

In distributed systems, we apply the same idea: each cell operates autonomously, with its own compute, database, and traffic. If one cell fails, the rest continue serving requests. This isolation reduces the blast radius and increases resilience.

This pattern became essential at Mercado Libre when the stock system powering Mercado Envíos reached its operational limit. Every inbound, outbound, reservation, and logistics movement depended on a single database that could no longer scale.

By early 2024, the question was clear:

What do you do when vertical scaling is no longer an option?

The problem: One database serving all of LATAM

Our initial architecture was simple: multiple stock services connected to a single MySQL cluster. This cluster managed:

  • Stock availability per Fulfillment Center
  • Reservations for Fulfillment shipments
  • Logistic movements across warehouses, sellers, and destinations

As volume grew, the pressure on MySQL intensified. By early 2024, we saw:

  • CPU and I/O usage constantly at maximum
  • Rising latency in critical operations
  • Degradation spikes that were unpredictable

Most importantly: a single failure could impact operations in multiple countries across the region.

Operating this way was no longer sustainable.

Image 1: Response time decreases ~70% ( — — before / — after).

Image 1: Response time decreases ~70% ( — — before / — after).

Why increasing the flavour wasn’t enough

For years, vertical scaling solved our problems. But eventually, we hit new limits:

  • There were no larger flavours available.
  • Traffic continued growing faster than capacity.
  • Relying on a single MySQL instance became a systemic risk.

Scaling was no longer about adding resources. It required rethinking our architecture.

Why we kept an intentional monolith

Some teams would split everything into microservices, but our domain required:

  • Extremely low latency, even under peak load
  • High availability — logistics can’t stop
  • Strong inventory consistency

Multiplying network hops or points of failure would have made these constraints harder. That’s why we kept an intentional monolith at the core of the system.

For us, the key wasn’t monolith versus microservices, but choosing the architecture the business needed. In our case, that meant a strong, centralized core domain, surrounded by isolated cells.

The solution: From one giant to multiple cells

We adopted a cell-based architecture to distribute load and isolate risks. Our model introduced two layers of segregation:

1. Cells by country groups

We grouped countries into three cells, balancing:

  • Traffic volume
  • Business importance
  • Resilience to failures

2. Read/write separation

We split flows into:

  • Writes: strictly consistent and business-critical
  • Reads: more flexible, backed by replicas and caching

Our goal remained simple: isolate, contain, and scale.

Image 2: Isolated cells reducing the blast radius.

Image 2: Isolated cells reducing the blast radius.

Fury: The platform that made cell operation predictable

Implementing cells isn’t just a technical challenge; it’s also an operational one.

How do you prevent each cell from becoming its own isolated universe with unique configs, dashboards, or deployment rules?

This is where Fury, Mercado Libre’s internal developer platform, became essential.

Fury allowed us to:

  • Create services and databases for each cell using consistent standards
  • Operate independent deployment pipelines per cell
  • Reuse routing, observability, and configuration components
  • Replicate patterns, so teams don’t have to reinvent them

Fury turned horizontal scaling into a repeatable, reliable process.

Routing challenges: When your APIs have no natural key

When we started designing routing logic for a cell-based architecture, we hit a structural limitation: several of our APIs simply didn’t include a natural key to identify which cell an entity belonged to.

Without that key, the traffic layer couldn’t deterministically route each request to the correct cell.

Introducing a new routing attribute wasn’t realistic. It would have required:

  • Breaking compatibility with existing clients
  • Updating public contracts used across the company
  • Coordinating changes with dozens of teams and services that depended on those APIs

In short, adding a routing key would have triggered a cross-organizational migration — one far more disruptive than the architectural change we were trying to implement. We needed a solution that would respect existing interfaces, even if those APIs weren’t designed for a sharded world.

The pragmatic solution: A broadcast gateway

When we realized several of our existing APIs had no routing key — and that introducing one would mean breaking contracts company-wide — we needed a fallback that worked without changing a single client.

For legacy entities, we implemented a pragmatic, backward-compatible routing fallback that allowed requests to be resolved without modifying existing clients. For new data, we adopted a deterministic ID strategy, so identifiers could serve as routing hints. Together, these approaches gave us a safe path to partition the dataset.

Flow

  1. The client makes the same request as always (no changes required).
  2. The traffic layer forwards the request to every cell.
  3. The cell holding the entity responds.
  4. If all cells return 404, the resource truly doesn’t exist.

It’s not the perfect architecture — broadcasting requests adds overhead — but it provided something more valuable: a safe migration path. It allowed us to gradually move to a cell-based model without forcing dozens of teams and clients to update their APIs overnight.

This balance of pragmatism and safety is what made the entire project viable.

ID management: Turning IDs into routing keys

Broadcasting solved the problem for legacy records, but we needed a deterministic routing mechanism for new entities — something that would scale cleanly as cells evolved and traffic grew.

The challenge was that our IDs had characteristics that worked against partitioning:

  • They were public, part of long-standing API contracts.
  • They were sequential, generated by a single MySQL instance.
  • They had no embedded meaning, so they couldn’t indicate which cell an entity belonged to.

This led to two major risks:

  • ID collisions across cells if two primaries generated overlapping numbers.
  • No way to change the ID structure without breaking every client integration.

The solution: Assigning ID ranges per cell

We introduced a simple but effective strategy: each cell was assigned its own exclusive ID range. Any new entity created in that cell would get an ID that could never collide with another cell.

This approach provided:

  • Collision-free generation, even with multiple independent primaries
  • Zero breaking changes, since IDs kept their original shape
  • Automatic routing, because the ID itself became the routing key
  • Full transparency for clients — they didn’t need to know anything about cells

With this setup, new records were easy to route, and old records were safely handled via broadcast. Using both strategies, we were able to migrate a massive, centralized system into a distributed, cell-based architecture without disrupting the business.

Migrating terabytes without stopping logistics

We migrated tens of terabytes of data into three independent cells using a carefully orchestrated cutover. Read traffic continued uninterrupted, and write operations were paused only for a brief, controlled window. Each cell’s full migration finished within a short, planned maintenance window.

During each cell’s cutover window, we executed the same playbook: copy the database into the target cell, briefly pause writes to ensure consistency, and keep all reads flowing from the original database so Marketplace and logistics traffic remained fully operational. Because most workflows are asynchronous, this short write downtime had minimal impact on end clients.

Once a cell received its data, we brought it online gradually. First, we enabled reads and validated critical flows, then allowed writes — protected by guardrails that prevented cross-cell contamination. After validation, we shifted read traffic to the new cell with zero downtime.

Each cell was migrated during its own controlled window, and all transitions completed successfully. In the year following migration, we observed a sustained reduction in infrastructure incidents and a significant improvement in platform availability.

High availability out of the box

Moving to a fully managed SQL service gave us a key capability we didn’t have before: replication with platform-backed consistency guarantees below 100 ms. This let us serve reads from replicas, eliminating reads on the primary, and dramatically improving scalability.

With automatic failover and replica management handled by the platform, our team no longer had to orchestrate infrastructure manually. This freed us to focus on evolving the stock domain, instead of maintaining the database.

Impact: What actually changed

A year after the migration, the results speak for themselves:

  • We reduced p95/p99 latency by ~70%.
  • We eliminated the recurring performance degradation caused by backup windows.
  • We shifted from scaling one giant database to scaling only the cells that need it.

In short, the migration didn’t just unlock performance. It removed a systemic risk and gave us an architecture we can safely grow on.

Image 3: Sharp latency drop immediately after cell split.

Image 3: Sharp latency drop immediately after cell split.

Lessons learned

Looking back, a few lessons stand out — not as abstract principles, but as things we experienced firsthand while reshaping a system under real constraints.

We learned that scaling isn’t just about throwing more resources at a problem; it’s about redistributing responsibility so no single component carries the weight of an entire region. We also saw the value of designing for routing from day one — a missing key can turn a simple refactor into a company-wide migration.

Sometimes, an intentional monolith is exactly what a domain needs, as long as the architecture around it is built to isolate failures and distribute load. Above all, we learned how a strong internal developer platform can turn an ambitious architectural shift into a repeatable, reliable path forward.

Each of these lessons came from solving problems we couldn’t afford to get wrong. They’ll resonate with any team standing at the edge of its current architecture.

Conclusion: Scale isn’t a bigger box. It’s many smaller ones

Migrating from a single MySQL instance to a cell-based architecture wasn’t just a technical refactor — it was a mindset shift. It meant challenging long-standing assumptions, carving a tens-of-terabytes monolith into independent pieces, and redesigning critical paths while the logistics engine of an entire region kept running. Hard? Absolutely. Worth it? Undeniably.

Today, we operate on an architecture that absorbs failures, scales where necessary, and behaves predictably under pressure. A year ago, a database spike could threaten operations in multiple countries. Now, a problem in one cell stays in that cell.

If your team feels “at the limit,” take this as an invitation:

Scaling doesn’t always mean making a single system bigger. More often, it means giving it room to multiply.

Not every architecture needs to become a giant. Sometimes, it needs to become a collection of smaller, safer, and faster pieces that unlock a future your old design could never reach.

Acknowledgments

This article was written with the support of Luram Archanjo.


메타데이터
post_id
528f581fb71b
slug
from-a-single-point-of-failure-to-a-cell-based-architecture-how-we-scaled-mercado-envíos-stock-528f581fb71b
url
https://medium.com/mercadolibre-tech/from-a-single-point-of-failure-to-a-cell-based-architecture-how-we-scaled-mercado-env%C3%ADos-stock-528f581fb71b
canonical_url
https://medium.com/mercadolibre-tech/from-a-single-point-of-failure-to-a-cell-based-architecture-how-we-scaled-mercado-env%C3%ADos-stock-528f581fb71b
author_url
https://medium.com/@rafael_silvestri
status
ok
fetched_at
2026-06-21 07:44:09