5 Students. Hindsight Remembers. CascadeFlow Decides. Our LLM Finally Gets Us.

The first version of our AI copilot was embarrassingly dumb — not because the model was bad, but because we gave it nothing to work with. Every session started cold. No memory of the user, no awareness of their goals, no recollection of corrections they’d made three times already. And to make it worse, we were routing a “what did I spend on coffee?” query to the same 70-billion-parameter model we used for multi-step budget forecasting.

We fixed both with two dedicated services: Hindsight for persistent user memory and CascadeFlow for adaptive model routing. This is how they work, why we separated them, and what we got wrong before we got it right.

The System in One Paragraph

UniFinance is a personal finance tracker built for international students. Expenses flow in from manual entry, OCR-scanned receipts, Gmail, and SMS. Everything lands in a PostgreSQL database on Supabase, accessible through a Next.js dashboard. On top of that sits an AI copilot — a chat interface that answers financial questions, generates budget recommendations, and tracks savings goals in natural language. That copilot is where all the interesting engineering decisions live.

System Architecture:

What Was Actually Broken

We had two failure modes that felt like one problem but weren’t.

Failure one: stateless AI: The LLM knew nothing about the user between sessions. If you corrected an OCR-predicted category — say, “Zomato” was classified as Miscellaneous when it should be Food — that correction existed only in the database. The AI didn’t know about it. Next conversation, same cold start. The copilot felt like talking to someone with no short-term memory.

Failure two: flat model routing: We sent every query to llama-3.3–70b-versatile regardless of complexity. A simple lookup — “what’s my total spending this month?” — hit the same model as “given my variable income and three active savings goals, should I increase my food budget?” That’s a 12x cost difference per million tokens ($0.59 vs $0.05) for tasks where the cheaper model would have been perfectly adequate.

Neither of these is hard to fix. What surprised us is how much both needed to be solved before either one worked well in practice.

Hindsight: Teaching the AI to Remember

Hindsight is an agent memory system — purpose-built for storing and retrieving long-term context per user. We use it for three categories of memory: user preferences, AI feedback history, and OCR correction rules.

The OCR correction path is the most concrete. When a user corrects a merchant category after an OCR scan, HindsightService.learnFromOcrCorrection() writes the override to a hindsight_memories table, keyed to that user and that merchant name:

OCR Receipt Pipe Line:

typescript

// hindsight.service.ts
async learnFromOcrCorrection(
  userId: string,
  merchant: string,
  originalCategory: string | null,
  updatedCategory: string
): Promise<boolean> {
  const cleanMerchant = merchant.trim();
  return this.updateMemory(userId, 'ocr_learning', cleanMerchant, {
    preferredCategory: updatedCategory,
    originalOcrCategory: originalCategory,
    lastUpdated: new Date().toISOString()
  });
}

The updateMemory call merges into an existing record if one exists — so the second time you correct “Zomato” to Food, it updates the same row rather than creating a duplicate. The correction is persistent, user-specific, and applied deterministically. No model involvement on the retrieval side. The next OCR scan for that merchant checks hindsight_memories before writing to the expenses table. Correct once, never correct again.

For the AI copilot, memories get injected into every prompt. Inside getFinancialContext(), Hindsight runs before the system prompt is assembled:

typescript

// financial-intelligence.service.ts
const memories = await hindsightService.getAllMemories(userId);
// ...serialized into the system prompt as:
// "Long-Term Memories (from Hindsight memory system): [...]"

The LLM receives the user’s goals, budget state, spending patterns, and all accumulated Hindsight memories in a single enriched system prompt. It doesn’t know what session number this is. It knows the user.

CascadeFlow: Routing That Earns Its Keep

CascadeFlow handles the routing layer. Each request is classified as SIMPLE, MEDIUM, or COMPLEX and sent to the appropriate model tier. The classification in askQuestion() is keyword-based and deliberately lightweight:

typescript

// financial-intelligence.service.ts
let complexity: TaskComplexity = 'MEDIUM';
if (
  qLower.includes('predict') ||
  qLower.includes('forecast') ||
  qLower.includes('afford') ||
  qLower.includes('should i') ||
  qLower.includes('optimize')
) {
  complexity = 'COMPLEX';
} else if (qLower.length < 35 && !qLower.includes('explain')) {
  complexity = 'SIMPLE';
}

SIMPLE routes to llama-3.1–8b-instant. MEDIUM and COMPLEX route to llama-3.3–70b-versatile. If the simple model fails or times out — CascadeFlow enforces a hard 15-second ceiling — it escalates automatically:

typescript

// cascadeflow.service.ts
if (options.complexity === 'SIMPLE' && model === this.models.SIMPLE) {
  console.log(`Escalating from ${this.models.SIMPLE} to ${this.models.MEDIUM}`);
  model = this.models.MEDIUM;
}

Every execution writes an audit record to ai_audit_logs in Supabase — model chosen, latency, token counts, estimated cost, escalation reason, success or failure. That table is not nice-to-have. It’s the only way we know the routing is actually behaving as designed. We found through those logs that queries with large Hindsight memory payloads escalate more often than the keyword classifier predicts. The injected context adds tokens; the 8b model has limits; the escalation policy handles it cleanly. But we wouldn’t have known without the logs.

How the Two Systems Connect

The two services never talk to each other directly. The orchestration lives entirely in FinancialIntelligenceService.askQuestion():

User Query

└─► getFinancialContext()

└─► hindsightService.getAllMemories() ← Hindsight fetches memories

└─► build enriched system prompt

└─► cascadeflowService.execute() ← CascadeFlow routes + logs

└─► classify complexity

└─► call model with 15s timeout

└─► [escalate on failure]

└─► write ai_audit_logs

└─► return answer

Hindsight’s job ends when it hands back memories. CascadeFlow’s job starts when it receives the enriched prompt. The financial intelligence service is the only thing aware of both.

We kept them separate by design. If Hindsight has a database connectivity issue, the copilot still answers — just without personalized context. If CascadeFlow’s model provider is degraded, we can fall back to a direct call on a fixed model without touching the memory system. Coupled together, one failure would take down both behaviors. Separated, they degrade independently and recover independently.

Three Things We’d Do Differently

Start with the audit log. We added ai_audit_logs midway through development. We should have built it first. You cannot tune a routing system you cannot observe. The cost-per-query data and escalation rate data were both surprising — in directions we couldn’t have guessed without measurement.

Account for injected context in complexity classification. Our keyword-based classifier looks at the raw user query. It doesn’t account for the Hindsight payload that gets appended to the system prompt. A short question with ten user memories injected is not a SIMPLE request. We’re moving toward factoring prompt token count into the routing decision.

Deterministic corrections beat probabilistic inference. For OCR correction rules, we store the override explicitly and apply it with a key lookup — not a fuzzy match, not a semantic similarity check. When the user says “this is Food,” that’s stored and applied exactly. No risk of the model “reinterpreting” the correction. Some things should not be probabilistic.

The architecture we arrived at — memory as a pre-execution layer, routing as an execution layer, a single orchestration service wiring them in sequence — is simpler to reason about than anything we tried before it. Each service improves on its own axis. Hindsight gets sharper as users make more corrections. CascadeFlow gets more accurate as we tune the routing thresholds. Neither needs to know the other exists.

That separation is what made this shippable. And honestly, it’s what made it interesting to build.