QuakePulse: Building a Real-Time Earthquake Dashboard with Aiven’s Free Tier

#AivenFreeTier

Every minute, somewhere on Earth, the ground shakes. Most of these tremors go unnoticed — small ripples deep beneath the ocean floor, micro-quakes in remote mountain ranges, aftershocks that only seismographs detect. But every so often, the Earth reminds us that we live on a restless planet. When it does, seconds matter.

QuakePulse is my attempt to turn raw seismic data into something anyone can see, understand, and react to — in real time.

Live Demo: https://quakepulse.onrender.com

Why I Built This

I’ve always been fascinated by how fast seismic data travels — from a fault rupture deep underground to a USGS data feed in under a minute. But that data sits in a JSON endpoint. Nobody stares at JSON.

I wanted to build something that makes the invisible visible: a live map where you can watch the Earth shake. Not a static chart updated once a day, but a truly streaming system where a magnitude 5.0 event in Chile appears on your screen seconds after the USGS records it.

At the same time, I wanted to learn streaming architecture properly — not by reading about it, but by building a real pipeline with real data flowing through it. When I saw Aiven’s free tier offering Kafka, PostgreSQL, and Valkey together, it clicked. That’s not just a database or a cache — that’s a complete data platform. I could build the entire streaming pipeline end-to-end without any infrastructure headaches.

The Architecture: Three Services, One Pipeline

The beauty of QuakePulse is that all three Aiven services play distinct, essential roles. None of them are bolted on for show — remove any one, and the system breaks.

Here’s how data flows:

1. A Kafka Producer polls the USGS earthquake feed every 60 seconds, deduplicates events by ID, and publishes them to three Kafka topics: raw-quakes (everything), significant-quakes (M4.5+), and quake-alerts (M6.0+ or tsunami warnings).
2. A Kafka Consumer reads from all three topics in parallel. For each earthquake, it performs a PostgreSQL upsert (for durable storage and analytics), caches it in a Valkey sorted set (for sub-second API responses), and publishes to a Valkey pub/sub channel for live push.
3. The WebSocket handler subscribes to Valkey’s live-quakes pub/sub channel and fans out new events to every connected browser — no polling, no delay.
4. The frontend renders earthquakes as color-coded circles on a dark-themed Leaflet.js map, with pulse animations for new arrivals and a Chart.js analytics panel.

Why These Three Services? The Architecture Decisions

Kafka: The Backbone You Don’t See

I could have skipped Kafka entirely — just poll USGS, write to PostgreSQL, done. But that would be a toy.

Kafka gives me decoupling. The producer doesn’t know or care what happens downstream. The consumer can crash, restart, and pick up exactly where it left off thanks to consumer group offsets. I can add new consumers later (email alerts, a Slack bot, a machine learning anomaly detector) without touching a single line of producer code.

I used three topics to implement priority routing at the infrastructure level:

• raw-quakes — every event, for completeness
• significant-quakes — M4.5+ events, for dashboards that only care about notable activity
• quake-alerts — M6.0+ or tsunami warnings, for critical notifications

Aiven’s free tier gives you 5 topics. I used 3 — enough to demonstrate the pattern without forcing unnecessary topics.

The Kafka producer also implements a circuit breaker: after 5 consecutive failures, it stops attempting sends for 60 seconds, preventing a cascade of timeout errors if the broker is temporarily unreachable.

PostgreSQL: The Source of Truth

Every earthquake that flows through the pipeline gets upserted into PostgreSQL using ON CONFLICT DO UPDATE. This means:

• First time we see an earthquake ID → INSERT
• USGS revises the magnitude or adds felt reports → UPDATE the existing row

PostgreSQL is the durable analytical store. The REST API queries it for historical data — earthquakes by time range, magnitude filterts, bounding box queries (for what’s visible on the map viewport), and aggregated statistics like hourly counts and magnitude distributions.

I deliberately chose not to use PostGIS. Bounding box queries with simple latitude BETWEEN ? AND ? comparisons are fast enough for this use case, and it avoids any dependency on extensions that might not be available on a free tier.

Valkey: The Speed Layer + Real-Time Glue

Valkey is doing double duty, and it’s the piece that ties the whole system together:

As a cache, it keeps the last 200 earthquakes in a sorted set (scored by timestamp). When the frontend loads or a user adjusts filters, the API hits Valkey first — sub-millisecond response times, no database round-trip. Individual earthquake details are cached with a 5-minute TTL. Daily stats are cached with a 60-second TTL.

As a pub/sub broker, it’s the real-time delivery mechanism. When the Kafka consumer processes a new earthquake, it publishes to the live-quakes channel. The FastAPI WebSocket handler subscribes to this channel and fans out the event to all connected browsers instantly.

This dual role is what makes Valkey indispensable in this architecture. Without it, I’d need either constant database polling (slow, wasteful) or a separate pub/sub system (over-engineered for this scale).

The Challenges I Faced (And How I Solved Them)

Challenge 1: SSL Everywhere

All three Aiven services require TLS. This is a good thing in production, but it means dealing with certificates from day one. Kafka needs a CA cert, service cert, and service key. PostgreSQL needs ?ssl=require on the connection string. Valkey uses the rediss:// scheme.

I centralized all of this in a Pydantic Settings config class that reads from environment variables and constructs SSL contexts programmatically. The Kafka SSL context is built once using Python’s ssl module, and the certs live in a certs/ directory that's .gitignored.

Lesson learned: Get the SSL plumbing right first. Everything else is easy after that.

Challenge 2: Graceful Degradation

What happens when Kafka is temporarily unreachable? Or Valkey restarts? In a demo, it’s tempting to pretend services never go down. But I wanted QuakePulse to degrade gracefully:

• If Kafka fails, the producer’s circuit breaker stops attempting sends and retries after a cooldown period.
• If Valkey is down, the API falls back to PostgreSQL for data and the WebSocket handler skips pub/sub.
• If the WebSocket connection drops on the client side, the frontend switches to REST polling every 15 seconds as a fallback, complete with a visual indicator showing “Polling” instead of “Live.”
• On startup, each service (DB, Kafka, Valkey) is initialized with a best-effort pattern — if one fails, the others still start, and the system runs in a degraded mode rather than crashing.

This makes the system resilient in a way that a typical demo project isn’t.

Challenge 3: Cold Start — The Empty Map Problem

On first launch, the USGS all_hour feed might only have 5-10 earthquakes. An almost-empty map isn't impressive.

The solution: on startup, QuakePulse seeds from the all_day.geojson feed — the last 24 hours of global seismicity. This typically gives 100-200 earthquakes immediately, so the map is populated with real data the moment you open it. After seeding, it switches to the regular 60-second polling loop for truly live updates.

Challenge 4: Deduplication Across Feeds

The USGS updates existing earthquakes (revised magnitudes, additional felt reports) which means the same earthquake ID can appear in multiple polls. The producer maintains an in-memory set of seen IDs (capped at 5,000 to prevent unbounded growth) to avoid re-publishing duplicates. On the database side, the ON CONFLICT DO UPDATE upsert ensures we always have the latest data without creating duplicate rows.

How Aiven’s Free Tier Enabled This

Let me be direct: I would not have built this project without Aiven’s free tier.

Setting up Kafka locally is notoriously painful — JVM dependencies, Zookeeper (or KRaft), broker configuration, topic creation. Setting up PostgreSQL isn’t bad, but adding proper TLS and async connection pooling adds friction. Valkey/Redis is the easiest of the three, but managing three separate local services while trying to build an application is a productivity killer.

With Aiven, I went from zero to fully connected in under 15 minutes:

1. Created three services in the Aiven console
2. Downloaded Kafka certificates
3. Copied connection URIs into my .env file
4. Ran uvicorn app.main:app — and data started flowing

No Docker Compose for infrastructure. No debugging “why won’t Kafka start.” No SSL certificate generation with openssl. Just production-grade managed services with a few clicks.

The free tier is generous enough for a real project too — 5 Kafka topics (I used 3), a proper PostgreSQL instance (not a toy SQLite), and an in-memory Valkey store fast enough for real-time caching and pub/sub. These aren’t sandboxed trial services with artificial limits — they’re real Aiven instances with real TLS, real availability, and the same API you’d use in production.

This let me spend my time on what matters: building the application, not fighting infrastructure.

The Frontend: Making Data Visceral

A streaming backend means nothing if the frontend doesn’t feel real-time. Here’s what makes QuakePulse’s map compelling:

• Dark-themed CartoDB tiles create a dramatic backdrop where earthquake markers pop visually.
• Color-coded circles scale by magnitude — green for micro-quakes, yellow for light, orange for moderate, red for strong, dark red for great (6.0+). You can see seismic patterns at a glance.
• Pulse animations — when a new earthquake arrives via WebSocket, its marker animates with a CSS pulse effect. Your eye is naturally drawn to the new event.
• Marker clustering — at low zoom levels, markers group into clusters labeled with counts, so the map stays readable even with hundreds of events.
• Click-to-fly — the side panel lists recent earthquakes in a scrollable list. Click any event and the map smoothly flies to that location.
• Connection indicator — a green dot means live WebSocket, yellow means fallback polling, red means disconnected. Transparency builds trust.

The entire frontend is vanilla JavaScript — no React, no build step, no node_modules. It loads fast, works everywhere, and the code is immediately readable.

What I’d Build Next

QuakePulse is a complete application, but the decoupled architecture means every extension is additive — new consumers, new features, zero changes to existing code.

• Location-based mobile notifications — users subscribe with their GPS coordinates and a radius (e.g., “alert me for anything M3.0+ within 200 km”). A new Kafka consumer on the quake-alerts topic computes haversine distances against subscriber locations and fires push notifications via Firebase Cloud Messaging or APNs. The subscriber list lives in PostgreSQL; the hot set of active subscriptions stays in Valkey for fast lookups. Because the alerting topic already exists, this entire feature is a new consumer — the producer and pipeline don’t change at all.
• Alert zones — draw a circle on the map and get browser notifications when a quake occurs in that area (client-side distance calculation, minimal server changes)
• More Kafka consumers — a Slack webhook consumer that posts to a channel when a significant quake hits, using the quake-alerts topic that's already producing messages
• Historical replay — use Kafka’s offset management to replay past events, effectively turning the dashboard into a seismic activity timelapse. Imagine scrubbing through a week of Pacific Ring of Fire activity in 30 seconds.

Try It Yourself

The entire project is open source. Clone it, provision three free Aiven services, add your connection URIs, and you’ll have a live earthquake dashboard in minutes.

GitHub: github.com/prabhu43/quakepulse

git clone https://github.com/prabhu43/quakepulse.git
cd quakepulse/backend
python -m venv venv && source venv/bin/activate
pip install -r requirements.txt
cp .env.example .env   # add your Aiven URIs
uvicorn app.main:app --host 0.0.0.0 --port 8000

Open http://localhost:8000 and watch the Earth shake.

Built for the Aiven Free Tier Competition. The Earth never stops shaking — now you can watch.

#AivenFreeTier