5 Students. Hindsight Remembers. CascadeFlow Decides. Our LLM Gets Us Now.

Every finance app we tried forgot us the moment we closed the tab. Correct a category once — it’s wrong again tomorrow. The AI copilot answers your question, then forgets everything by the next session. We built UniFinance because we were tired of starting from zero every time, and we solved it by wiring two systems together: Hindsight for long-term memory and CascadeFlow for adaptive model routing.

What We Built

UniFinance is a personal expense tracker built for students managing money across multiple channels simultaneously. Transactions flow in from manual entry, scanned receipts, SMS notifications, and Gmail integrations. Everything reconciles into a PostgreSQL ledger on Supabase, surfaced through a Next.js dashboard.

1. System Architecture

The stack is deliberately split: a Next.js frontend, Supabase for auth and storage with Row-Level Security, and a FastAPI microservice running PaddleOCR for receipt scanning. The AI copilot lives on top — and that’s where the interesting architectural decisions live.

The Core Problem: Two Failure Modes at Once

We had two separate problems that looked like one problem.

Problem one: The LLM had no memory across sessions. A user who’d corrected “Swiggy” from Misc to Food three times last week had to keep correcting it. A user who’d explicitly told the copilot they prioritize a travel goal over cutting food spending would get a generic “reduce dining expenses” response on the next visit. The context just wasn’t there.

Problem two: We were routing every query to the same model regardless of complexity. “How much did I spend on food?” is a lookup. “Given my income fluctuates between ₹40,000–₹65,000 and I have three active savings goals, what’s a realistic allocation strategy for next month?” is multi-step reasoning. Routing both to llama-3.3-70b-versatile wastes tokens on the simple one. Routing both to llama-3.1-8b-instant returns incomplete answers on the complex one.

2. Hindsight Memory & CascadeFlow LLM Routing

We could have hacked around both problems inside a single “AI service.” We didn’t. We built them as two separate, independently failing services and wired them in sequence inside FinancialIntelligenceService.

Hindsight: Memory as a Pre-Execution Layer

Hindsight is an agent memory system that stores and retrieves long-term user context. In UniFinance, HindsightService manages three categories of memory: user preferences, AI feedback history, and OCR correction rules.

The OCR correction path is where it earns its keep most visibly. When a user corrects a category after an OCR scan, we write that correction directly to the hindsight_memories table keyed to that merchant:

typescript

// hindsight.service.ts
async learnFromOcrCorrection(
  userId: string,
  merchant: string,
  originalCategory: string | null,
  updatedCategory: string
): Promise<boolean> {
  const cleanMerchant = merchant.trim();
  return this.updateMemory(userId, 'ocr_learning', cleanMerchant, {
    preferredCategory: updatedCategory,
    originalOcrCategory: originalCategory,
    lastUpdated: new Date().toISOString()
  });
}

The updateMemory method merges into an existing record if one exists — so repeated corrections for the same merchant accumulate rather than overwrite. No retraining. No model changes. The correction is stored and applied deterministically on the next scan.

For the AI copilot, memories get injected into every prompt as structured context. Inside getFinancialContext(), Hindsight runs before anything else:

typescript

// financial-intelligence.service.ts
const memories = await hindsightService.getAllMemories(userId);
// ...memories are then serialized into the system prompt

By the time CascadeFlow sees the request, the system prompt already contains the user’s full behavioral history — goals, feedback patterns, OCR correction rules — all of it.

CascadeFlow: Routing That Pays for Itself

CascadeFlow handles adaptive model routing. The logic in cascadeflow.service.ts is direct: classify the request complexity, pick the right model, enforce a 15-second timeout, and escalate automatically on failure.

typescript

// cascadeflow.service.ts
private models = {
  SIMPLE: 'llama-3.1-8b-instant',
  MEDIUM: 'llama-3.3-70b-versatile',
  COMPLEX: 'llama-3.3-70b-versatile',
};

// Auto-escalate to balanced model if lightweight fails
if (options.complexity === 'SIMPLE' && model === this.models.SIMPLE) {
  model = this.models.MEDIUM;
}

The pricing delta makes this worth engineering: llama-3.1-8b-instant costs $0.05 per million prompt tokens. llama-3.3-70b-versatile costs $0.59 — nearly 12x more. For simple queries that make up the bulk of daily usage, the routing pays for itself quickly.

Complexity is determined in askQuestion() based on keyword signals:

typescript

// financial-intelligence.service.ts
let complexity: TaskComplexity = 'MEDIUM';
if (qLower.includes('predict') || qLower.includes('forecast') ||
    qLower.includes('afford') || qLower.includes('should i') ||
    qLower.includes('optimize')) {
  complexity = 'COMPLEX';
} else if (qLower.length < 35 && !qLower.includes('explain')) {
  complexity = 'SIMPLE';
}

Pipeline 1: OCR Receipt Ingestion Pipeline

Every execution — model chosen, latency, token counts, estimated cost, escalation reason — gets logged to ai_audit_logs in Supabase. That audit trail is how we know the routing is actually working. We discovered through those logs that queries carrying large Hindsight memory payloads escalate more often than the keyword classifier predicts. The injected context pushes token counts up, which in turn pushes the 8b model past its reliable range. The escalation policy catches it; the audit log tells us why.

How They Wire Together

The sequence inside FinancialIntelligenceService is the key architectural decision:

User Query
  → getFinancialContext()
      → hindsightService.getAllMemories()     // Hindsight runs first
      → analyticsService.getAnalyticsSummary()
      → build enriched system prompt
  → cascadeflowService.execute()             // CascadeFlow runs second
      → classify complexity
      → route to model
      → [escalate on failure]
      → log audit record
  → return answer to user

The two services never communicate directly. Hindsight’s job is done when it returns memories. CascadeFlow’s job starts when it receives the enriched prompt. FinancialIntelligenceService is the only thing that knows both exist.

Pipeline 2: SMS Integration Ingestion Pipeline

Pipeline 3: Gmail Integration Ingestion Pipeline

This separation was a deliberate call. Hindsight failing means the copilot answers without personalization — degraded, but functional. CascadeFlow failing throws an error we can handle with a fallback. If they were coupled, one outage takes down both behaviors. Decoupled, they degrade independently.

Lessons We’d Apply Again

Separate memory from routing. They have different failure modes, different scaling concerns, and different improvement curves. Collapsing them into one service makes both harder to reason about.

Audit every LLM call. CascadeFlow logging model, latency, and escalation reason to Supabase wasn’t optional — it was the only way we knew the routing was behaving as expected. Without it, we’d be guessing.

Deterministic corrections beat probabilistic inference. For OCR merchant corrections, we store the override explicitly and apply it with a direct key lookup. We don’t ask the LLM to infer from examples. When a user says “this is Food, not Misc,” that’s what gets stored and applied. No hallucination risk on the correction path.

OCR preprocessing is not optional. An earlier version skipped CLAHE contrast normalization. Receipts with uneven lighting came back with garbled merchant names and wrong totals. The 80ms preprocessing cost was immediately worth it.

Context length changes your complexity classification. A short question with ten injected Hindsight memories is not a simple request. Account for total prompt size when routing, not just the raw query length.

The architecture we landed on — memory as a pre-execution layer, routing as an execution layer, a single orchestration service connecting them — turned out to be more composable than we expected. Each service improves independently. Neither needs to know the other exists. That separation is what made the system actually shippable.

This project wouldn’t exist without the four people who built it alongside me — Vaishnavi Anugu, Lakshsutle, Mummadi Siddeshwar, and Burragoutham. Every architectural decision in this write-up was a conversation, every late-night debug was shared, and every shipped feature had all five of us behind it.