Building a Context-Aware Financial News Chatbot

Retrieval-augmented generation (RAG) using cosine similarity over embeddings, connected to a Gemini financial analyst persona.

December 2023 · 7 min read RAG PGVector Gemini FastAPI Python NEPSE

Once you have a database of 768-dimensional news embeddings linked to NEPSE companies, the natural next step is: let users talk to it. This post covers how the RAG (retrieval-augmented generation) chatbot works — from the time-aware query parser, through cosine similarity retrieval, to the Gemini response with a financial analyst persona.

Code and full pipeline: github.com/Srmaraghu/news-data-pipeline

The Core Problem with LLM-Only Financial Chatbots

If you just ask Gemini “What’s the latest news on NABIL Bank?”, you get hallucinated financial news from its training data. The training cutoff means anything recent is fabricated — and worse, it sounds confident while doing it.

The fix is retrieval-augmented generation: search your own database for relevant articles first, then ask the model to answer using only what you retrieved. The model becomes a synthesizer, not a recall machine.

User query: "latest news on NABIL last 3 days"
    │
    ▼
┌────────────────────────────────────────┐
│ Query Parser                            │
│  extract: date_filter, hour_filter      │
│           prioritize_recency flag       │
└──────────────────┬───────────────────────┘
                    │
                    ▼
┌────────────────────────────────────────┐
│ Embed query (Gemini, 768-dim)           │
└──────────────────┬───────────────────────┘
                    │
                    ▼
┌────────────────────────────────────────┐
│ Cosine similarity search                │
│  post-filter: date / hour / recency     │
└──────────────────┬───────────────────────┘
                    │
                    ▼
┌────────────────────────────────────────┐
│ Prompt assembly                         │
│  1. system instruction (analyst persona)│
│  2. history clause (last 3 turns)       │
│  3. top 30 articles (title+date+body)   │
│  4. time clause (explicit date range)   │
│  5. user query                          │
└──────────────────┬───────────────────────┘
                    │
                    ▼
              Gemini generate

Step 1: Time-Aware Query Parsing

Before touching the database, the system parses the user’s query for time signals:

def extract_query_info(query):
    query_lower = query.lower()
    date_filter = None
    time_phrase = None
    prioritize_recency = False
    hour_filter = None

    if any(word in query_lower for word in ["latest", "recent", "newest", "today"]):
        prioritize_recency = True

    hour_match = re.search(r'last (\d+) hour', query_lower)
    if hour_match:
        hours = int(hour_match.group(1))
        hour_filter = current_time - timedelta(hours=hours)
        time_phrase = f"last {hours} hour(s)"
        prioritize_recency = True

    # ...logic for days/yesterday
    return date_filter, time_phrase, prioritize_recency, hour_filter

This extracted intent — not just the raw query string — is what drives the post-retrieval filtering in Step 3.

Step 2: Embedding the Query + Cosine Similarity Search

The user’s query is embedded using the same 768-dimensional Gemini model used during article ingestion. This consistency matters: query and document embeddings must come from the same model for cosine similarity to be meaningful. Mixing embedding models — even ones with the same dimensionality — produces vectors that don’t share a coordinate space, and similarity scores become noise.

query_embedding = get_gemini_embedding(gemini_client, query)
query_embedding = np.array(query_embedding, dtype=np.float32)

def cosine_similarity(vec1, vec2):
    vec1, vec2 = np.array(vec1, dtype=np.float32), np.array(vec2, dtype=np.float32)
    if np.linalg.norm(vec1) == 0 or np.linalg.norm(vec2) == 0:
        return 0.0
    return np.dot(vec1, vec2) / (np.linalg.norm(vec1) * np.linalg.norm(vec2))

similarities = []
for item in data:
    combined_embedding = json.loads(item.get("combined_embedding"))
    similarity = cosine_similarity(query_embedding, combined_embedding)
    similarities.append((similarity, item))

similarities.sort(reverse=True, key=lambda x: x[0])
top_results = similarities[:30]

The current implementation loads all embeddings from the database and computes similarity in pure Python. This works fine at the current scale — tens of thousands of articles — but it’s an O(n) brute-force scan, and it won’t hold up at millions of rows. The PGVector extension supports native approximate nearest-neighbor search via an IVFFLAT index, which would push this similarity computation down into Postgres itself. Migrating to that when the dataset outgrows brute-force search is a straightforward swap, since the embedding storage format doesn’t need to change — only the query path does.

Step 3 & 4: Post-Retrieval Filtering and Conversation History

After similarity ranking, additional filters are applied. Similarity and recency are sometimes in tension: an article from three days ago might be semantically closer to the query than one from this morning, simply because it uses more similar phrasing. The prioritize_recency flag — set during query parsing — overrides the pure similarity sort whenever the user explicitly asks for "latest" or "recent" content, trading semantic closeness for time-relevance.

Every chat session is stored per user, and the last 3 conversation turns are loaded and included in the prompt. Using the last 3 turns instead of the full history is a deliberate tradeoff: it keeps the context window bounded regardless of how long a session runs, at the cost of the model losing thread on anything mentioned earlier than that.

Step 5: Prompt Assembly

The final prompt is assembled from four sections, in order:

1. System instruction — the financial analyst persona.
2. History clause — the last 3 conversation turns.
3. Article content — the top 30 retrieved articles (title, date, body).
4. Time clause — an explicit note about the time period being discussed.
5. User query.

Putting the time clause directly before the user query — rather than burying it in the system instruction — reinforces to the model which time window it should reason within, since instructions closer to the query tend to get weighted more heavily in practice.

The Stock-Specific Recommendation Pipeline

The chatbot handles general news queries through the RAG flow above. But for stock-specific lookups — when a user searches for NABIL directly — a separate pipeline kicks in, running concurrent calls via asyncio.gather:

recommendations_result, sentiments = await asyncio.gather(
    get_recommendations(),  # Gemini: news + technicals → Buy/Hold/Sell
    get_sentiments(),       # Gemini: article list → [positive, negative, neutral, ...]
    return_exceptions=True
)

return_exceptions=True is the important detail here: if one call fails (say, the sentiment call times out), the other still completes instead of the whole gather aborting. The caller is responsible for checking which results came back as exceptions.

Live market data is fetched from OnlineKhabar’s market API across 7 endpoints concurrently. The technical indicators fed to Gemini for the recommendation include MACD, RSI, MFI, CCI, MA20/50/200, ADX, P/E ratio, and Beta — giving the model both narrative (news, sentiment) and quantitative (technicals) signal to reason over before producing a Buy/Hold/Sell call.

What the System Gets Right

• Time-aware retrieval. A query for “news yesterday” returns only yesterday’s articles, because the time filter is applied as a hard post-filter, not left to the model’s judgment.
• Multi-source synthesis. The recommendation signal draws from news sentiment, sector trends, and technical indicators rather than any single source.
• Explainability. Every signal includes a plain-English rationale citing specific data points, so the output isn’t just a label — it’s a traceable judgment.

Part of a series on building Nepali financial news intelligence platform. See also . How I Orchestrated 14 Concurrent News Scrapers