I Built an AI That Finds Root Cause of Production Incidents in Seconds — Here’s How

When production breaks at 2 AM, every engineer opens four tabs — GitHub, Sentry, Slack, and deployment logs — and spends 45 minutes manually piecing together what went wrong.

I built ShadowOps to fix that. ━━━━━━━━━━━━━━━━━━━━━━

🔥 THE PROBLEM

Picture this: your checkout API is returning 504 errors. 12,500 users can’t complete purchases. Your phone is blowing up.

You check GitHub — was there a recent PR? You check Sentry — what’s the error? You check Slack — what is the team saying? You check your deployment pipeline — what changed?

Four tools. Four logins. One very stressful engineer.

This is the daily reality for on-call engineers at every company running microservices at scale.

━━━━━━━━━━━━━━━━━━━━━━

💡 THE SOLUTION — SHADOWOPS

ShadowOps is an AI-powered incident intelligence platform that automatically connects all your data sources and delivers root cause analysis in seconds — not hours.

Here is what it does:

✅ Cross-Source Timeline Merges GitHub PRs, Sentry errors, Slack messages, and deployment events into one unified chronological view.

✅ AI Root Cause Analysis Delivers confidence-scored analysis with a step-by-step reasoning chain, each step citing a specific source and evidence.

✅ Natural Language Query Ask “Why did the checkout API fail?” and get a generated SQL query plus an AI answer backed by real cross-source data.

✅ Similar Incident Detection Finds past incidents with matching patterns so your team learns from history instead of repeating it.

━━━━━━━━━━━━━━━━━━━━━━

🧠 THE CORE INNOVATION — CORALQUERYENGINE

The heart of ShadowOps is the CoralQueryEngine — inspired by Coral’s federated SQL concept.

Traditional approach (slow): → Query GitHub API → Query Sentry API → Query Slack API → Query deployment system → Manually merge all data → Try to find a pattern

ShadowOps approach (instant): → One CoralQueryEngine call → All 4 sources joined automatically → AI analyzes the merged result

The engine simulates this SQL across live data sources:

SELECT * FROM github JOIN sentry ON sentry.incident_id = github.incident_id JOIN slack ON slack.incident_id = github.incident_id JOIN deployments ON deployments.incident_id = github.incident_id ORDER BY timestamp DESC

It even translates natural language into this SQL automatically. Ask “Why did checkout fail?” — it detects keywords like “deploy” and “error”, selects the right sources, and builds the query.

━━━━━━━━━━━━━━━━━━━━━━

🛠️ TECH STACK

Frontend → Angular 17 (Standalone Components) Styling → Tailwind CSS (Dark SaaS UI) Backend → Python FastAPI Schemas → Pydantic v2 Data → JSON mock files (zero dependencies) AI Layer → Prompt-based reasoning engine

━━━━━━━━━━━━━━━━━━━━━━

📐 ARCHITECTURE

The system has three main layers:

Layer 1 — Angular Frontend Three pages: Dashboard, Incident Detail, and Query Page. Built with standalone components, lazy-loaded routes, and a dark Tailwind CSS theme styled like Datadog or Linear.

Layer 2 — FastAPI Backend Clean layered architecture with routers, services, models, and data layers. Five REST endpoints cover everything from incident listing to AI analysis.

Layer 3 — CoralQueryEngine The intelligence core. Reads from four JSON data sources, performs cross-source correlation, builds timelines, and powers the natural language query system.

━━━━━━━━━━━━━━━━━━━━━━

🎬 THE DEMO — REAL INCIDENT WALKTHROUGH

I built three realistic incident scenarios into the system. Let me walk through the most dramatic one.

INCIDENT INC-001: Checkout API Critical Failure 12,500 users affected · 133 minutes downtime · $265K estimated loss

Here is what the cross-source timeline revealed:

13:45 → [GitHub] PR #232 merged — scaled replicas from 3 to 15 for Black Friday traffic

14:02 → [GitHub] PR #234 merged — reduced DB connection pool from 100 to 20 to save memory

14:05 → [Deployment] v2.3.1 deployed — bundled BOTH PRs together

14:32 → [Sentry] DatabaseConnectionTimeout × 15,234 errors “Pool size: 20, Active: 20, Idle: 0, Waiting: 847”

14:38 → [Slack] sarah.chen: “Found it — PR #234 reduced pool to 20 WHILE PR #232 scaled replicas to 15. 15 pods × 8 connections = 120 needed vs 20 available.”

14:52 → [Deployment] Emergency rollback triggered

The AI analyzed all 14 events across all 4 sources and returned:

Root Cause: Two PRs with conflicting assumptions deployed simultaneously. Confidence: 96%.

Fix: Restore pool size immediately. Long-term: pool_size must always be calculated as max_replicas × connections_per_pod × 1.3.

━━━━━━━━━━━━━━━━━━━━━━

🚀 HOW TO RUN IT

Clone the repository

git clone https://github.com/YOUR_USERNAME/shadowops

Start backend

cd backend python -m venv venv venv\Scripts\activate pip install -r requirements.txt uvicorn app.main:app — reload — port 8000

Start frontend

cd frontend npm install npm start

Open http://localhost:4200

No API keys needed. No external services required. Everything runs locally with realistic mock data.

━━━━━━━━━━━━━━━━━━━━━━

📚 KEY LESSONS LEARNED

1. The CoralQueryEngine pattern is genuinely powerful Federated cross-source queries solve a real problem that every engineering team faces. The concept translates beautifully from Coral’s distributed SQL to application-level incident correlation.

2. Mock data quality matters more than you think Realistic mock data — with actual timestamps, real error messages, and genuine Slack conversation patterns — makes a demo feel like production. Judges and users notice.

3. Angular 17 standalone + Tailwind is the fastest premium UI stack No NgModules, no complex setup. Standalone components with lazy loading and Tailwind’s dark utility classes produced a Datadog-quality interface in hours.

4. AI without an LLM API is possible for demos Template-driven reasoning with pre-crafted analyses per incident scenario works perfectly for hackathon demos and lets the system run completely offline.

━━━━━━━━━━━━━━━━━━━━━━

🔗 LINKS

🔗 GitHub:https://github.com/kaushalkarkar/shadowops-coral 🎬 Demo:https://youtu.be/bTLQxWSihpU

━━━━━━━━━━━━━━━━━━━━━━

If you are building developer tools or incident management systems, I would love to hear your thoughts.

What data sources would you add to ShadowOps next? Drop a comment below 👇

Built for hackathon · CoralQueryEngine™