Legacy to AI: A Practical Blueprint for Platform Modernization in Financial Services

How I built a RAG-powered retirement advisor that could save wealth management firms $800K+ annually — without replacing their existing systems

The RMD Advisor chat interface answering a customer question

Every December, financial advisors across America brace for the RMD tsunami.

Thousands of retirees suddenly remember they need to withdraw money from their retirement accounts before year-end — and they all have questions. Complex questions about SECURE Act 2.0 changes, inherited IRA rules, spousal beneficiary exceptions, and Qualified Charitable Distribution strategies.

The phone lines jam. Email queues overflow. Advisors spend hours explaining the same rules, over and over, while customers wait days for callbacks.

What if AI could handle 50% of these inquiries instantly, accurately, and 24/7 — saving over $800,000 annually while actually improving customer satisfaction?

I built it. Here’s the blueprint.

AI orchestration layer wrapping legacy Java calculation engine

The Business Problem Nobody Talks About

Required Minimum Distributions (RMDs) might sound like an obscure tax technicality, but they represent one of the most critical touchpoints in wealth management. Every American with a traditional IRA or 401(k) must start taking mandatory withdrawals at a certain age — and getting it wrong means a 25% IRS penalty.

The SECURE Act 2.0, passed in late 2022, made everything more complicated:

• The age rules changed. RMDs now start at 73 for people born between 1951–1959, and 75 for those born in 1960 or later. Millions of retirees are confused about which rule applies to them.
• Inherited IRA rules became byzantine. The 10-year rule, eligible designated beneficiaries, required beginning dates — even seasoned advisors need to look up the specifics.
• The penalty dropped from 50% to 25%. Good news, but it created a wave of questions about correcting past mistakes.

For firms like Charles Schwab, Fidelity, and Vanguard — managing trillions in retirement assets — this translates to a massive operational challenge:

• High volume: Hundreds of thousands of RMD-related inquiries per year
• High stakes: Wrong guidance could cost customers thousands in penalties
• High complexity: Each situation involves multiple variables
• Concentrated demand: Most questions come in October-December

The traditional solution? Hire more advisors. Train them extensively. Hope they don’t burn out during the year-end crunch.

There has to be a better way.

Why Traditional Approaches Fall Short

Before diving into AI, I spent time understanding why existing solutions don’t work:

Static FAQs and knowledge bases can’t handle the combinatorial complexity. “When do RMDs start?” has at least four different answers depending on birth year, account type, and whether you inherited the account. A customer asking about their specific situation gets lost clicking through nested pages.

Rule-based chatbots break down quickly. RMD logic involves too many conditional branches, exceptions, and cross-references. Maintaining decision trees becomes a nightmare, and customers hate the “I don’t understand, please rephrase” responses.

Hiring more advisors doesn’t scale economically. Training someone to handle complex RMD questions takes months. Turnover during stressful year-end periods is high. And you’re paying premium rates for people to answer the same questions repeatedly.

Existing calculation engines work great — but they’re trapped in legacy systems. Financial institutions have spent years building precise, audited, compliant calculation logic in Java or .NET. That code is bulletproof. But it speaks in numbers, not natural language.

The insight that shaped my approach: Don’t replace what works. Augment it with intelligence.

The Architecture: AI That Respects What Already Exists

Here’s what I built — an AI layer that wraps existing calculation engines with natural language understanding:

The key architectural decisions:

1. RAG (Retrieval-Augmented Generation) for knowledge. Instead of fine-tuning an LLM on RMD rules — which would be expensive and hard to update — I built a vector database from authoritative sources. When a customer asks a question, the system retrieves the most relevant IRS regulations and uses them to generate an accurate response.

2. Java for calculations, Python for AI. The calculation engine stays in Java because that’s where proven financial logic lives. The AI layer handles natural language understanding and orchestration. They communicate via REST APIs — a clean separation that respects organizational boundaries.

3. Gradio for rapid prototyping. For stakeholder demos and user testing, Gradio lets me build a functional chat interface in hours, not weeks. In production, this would connect to existing web and mobile applications.

4. Local embeddings for privacy. The embedding model runs locally using sentence-transformers. Customer questions never leave the infrastructure for vectorization — only the final query goes to the LLM, and even that can be self-hosted if needed.

What this architecture enables:

• ✅ Preserves investment in proven calculation logic
• ✅ Adds intelligence without rewriting core systems
• ✅ Enables gradual adoption (start with simple queries, expand over time)
• ✅ Creates an API gateway for future integrations (mobile, CRM, voice)

The Business Case: $800K+ Annual Savings

Let’s talk numbers — because AI projects that can’t demonstrate ROI don’t get funded.

Conservative assumptions:

• Annual RMD-related inquiries: 100,000
• AI deflection rate: 50%
• Average advisor handling time: 20 minutes
• Fully-loaded advisor cost: $50/hour
• LLM cost per query: $0.01

The math:

Calls deflected to AI:     100,000 × 50% = 50,000 calls
Advisor time saved:        50,000 × 20 min = 16,667 hours
Labor cost avoided:        16,667 hours × $50 = $833,350
AI operating cost:         100,000 queries × $0.01 = $1,000
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Net annual savings:        $832,350
ROI:                       83,235%

Beyond the direct savings:

• Customer satisfaction improves because answers are instant, not 24–48 hour callbacks
• Advisor satisfaction improves because they handle interesting cases, not repetitive questions
• Compliance risk decreases because AI gives consistent, accurate guidance every time
• Year-end capacity crisis disappears because AI scales infinitely

Even if you cut these estimates in half, the ROI remains compelling.

Precise RMD calculation with IRS table reference and explanation

And unlike hiring, the marginal cost of handling additional queries approaches zero.

Technical Implementation: Key Decisions

I won’t turn this into a tutorial — you can explore the full code on GitHub. But here are the decisions that mattered:

LangChain with LCEL (LangChain Expression Language)

LangChain 0.2+ introduced a composable, pipe-based syntax that makes RAG chains more maintainable:

chain = (
    {"context": retriever | format_docs, "question": RunnablePassthrough()}
    | prompt
    | llm
    | StrOutputParser()
)

This replaced the deprecated RetrievalQA pattern that many tutorials still show. If you're building RAG in 2025, use LCEL—it's more explicit and easier to debug.

ChromaDB for vector storage

I chose ChromaDB over Pinecone or Weaviate for this project because:

• It runs locally with zero configuration
• No vendor lock-in or API costs during development
• Easy to swap for a managed solution in production
• The 2201 chunks from IRS documents fit comfortably in memory

Google Gemini as the LLM

Gemini offers a generous free tier that’s perfect for development and demos. The quality is comparable to GPT-4 for this use case, and the cost at scale is significantly lower. The architecture supports OpenAI as a fallback — LLM choice shouldn’t be a one-way door.

Spring Boot for the calculation service

The Java service implements the IRS Uniform Lifetime Table with exact values from Publication 590-B. This isn’t AI — it’s precise arithmetic that’s been done correctly in financial systems for decades. The AI layer calls this service when it needs a calculation, keeping responsibilities cleanly separated.

Lessons Learned

What worked better than expected:

RAG with authoritative sources produces trustworthy answers. When the LLM generates a response grounded in actual IRS publications, it’s remarkably accurate. The key is curating the knowledge base carefully — garbage in, garbage out.

Handling a complex inherited IRA question with source citations

Java + Python integration was smooth. I expected friction at the language boundary, but REST APIs make it trivial. Each service can be developed, tested, and deployed independently. This pattern works.

Customers don’t need to know it’s AI. The best compliment for an AI system is when users simply say “that was helpful.” They don’t need to see the machinery — they need their questions answered.

What surprised me:

Prompt engineering matters more than model selection. Switching from GPT-4 to Gemini had less impact than refining my system prompt. The difference between a good and great RAG prompt is 10+ hours of iteration.

The knowledge base requires ongoing curation. IRS rules change. New guidance gets published. A RAG system isn’t “build once and forget” — it needs a refresh process. Budget for this.

What I’d do differently in production:

Add response caching. Many questions are nearly identical. A semantic cache that recognizes similar queries could cut LLM costs by 60–70% and improve latency.

Implement guardrails. For a financial application, I’d add explicit checks to prevent hallucination on numerical values. The LLM should retrieve and cite, never calculate.

Build an evaluation framework. Before deploying, I’d create a test suite of 200+ question-answer pairs validated by domain experts. Automated regression testing is non-negotiable for production AI.

The Bigger Picture: AI-Driven Platform Modernization

This project demonstrates a pattern that applies far beyond RMD calculations:

Legacy systems contain enormous value — years of domain expertise encoded in code, tested edge cases, regulatory approvals. Throwing them away to “go AI-native” destroys value and creates risk.

AI is most powerful as an augmentation layer. It handles the messy, unstructured, natural-language interface with humans. The legacy system handles the precise, audited, compliant business logic. Together, they’re better than either alone.

The business case must drive the architecture. I didn’t build this because RAG is cool. I built it because financial services firms have a real, expensive, frustrating problem that AI can solve. Start with the problem, not the technology.

Where This Goes Next

The RMD Advisor demonstrates core capabilities. A production deployment would add:

• GraphRAG for better handling of interconnected regulations
• Multi-turn conversation memory for complex advisory sessions
• Integration with account systems for personalized calculations
• Kubernetes deployment for enterprise-scale reliability
• Comprehensive evaluation framework for quality assurance

The architecture is ready. The business case is proven. The question is: who will be first to deploy?

Let’s Connect

I’m currently exploring opportunities in AI-driven platform modernization, particularly in financial services and regulated industries. If your organization is tackling similar challenges — modernizing legacy systems, implementing production RAG, or building AI that solves real business problems — I’d welcome a conversation.

🔗 Project Repository: github.com/alexey-tyurin/rmd-advisor

💼 LinkedIn: linkedin.com/in/alexey-tyurin-36893287

📧 Email: altyurin3@gmail.com

Alexey Tyurin is a Staff Software Engineer with 15+ years of experience building scalable systems at PayPal and other enterprises. He specializes in AI-driven platform modernization, RAG architectures, and bridging legacy systems with modern AI capabilities. Google Cloud Certified Professional Machine Learning Engineer.