Building a Real-Time AI Developer Assistant with Cohere, FastAPI, and Angular

Taking our GenAI app to the next level with streaming responses and code highlighting

Introduction

In my previous article, I walked through how to use Cohere’s free API to build a simple developer assistant API.

But a static response is only the beginning.

In this article, we’ll take things to the next level by building a real-time AI-powered chat application with:

• ✅ Streaming responses (ChatGPT-style typing)
• ✅ Syntax-highlighted code blocks
• ✅ Angular frontend + FastAPI backend
• ✅ Clean, production-like architecture

By the end, you’ll have a mini ChatGPT-style developer assistant running locally.

Final Architecture

Angular (UI)
   ↓
FastAPI (Streaming API)
   ↓
Cohere Chat API

Why Streaming Matters

In the first version, the flow was:

User → API → Wait → Full response

Now, we improve it to:

User → API → Stream chunks → UI updates in real time

This creates a much better user experience and mimics modern AI tools.

Backend: FastAPI + Cohere Streaming

Step 1: Install Dependencies

pip install fastapi uvicorn cohere sse-starlette python-dotenv

Step 2: Streaming Endpoint

Cohere provides a streaming API, which we expose using Server-Sent Events (SSE).

from fastapi import FastAPI, Query
from sse_starlette.sse import EventSourceResponse
import cohere, os

co = cohere.Client(os.getenv("COHERE_API_KEY"))

app = FastAPI()

@app.get("/ask-stream")
async def ask_question_stream(question: str = Query(...)):

    async def event_generator():
        stream = co.chat_stream(
            model="command-r-08-2024",
            message=question
        )

        for event in stream:
            if event.event_type == "text-generation":
                yield {"data": event.text}

    return EventSourceResponse(event_generator())

Why We Use GET Instead of POST

This is important.

The browser API used for streaming:

new EventSource(url)

✅ Supports only GET ❌ Does NOT support POST

So even though our normal API uses POST:

@app.post("/ask")

Streaming must use:

@app.get("/ask-stream")

Frontend: Angular Streaming + Rendering

Step 1: Streaming with EventSource

Angular’s HttpClient does not support SSE, so we use:

askStreaming(
  question: string,
  onMessage: (chunk: string) => void,
  onComplete: () => void
) {
  const url = `http://localhost:8000/ask-stream?question=${encodeURIComponent(question)}`;

  const eventSource = new EventSource(url);

  eventSource.onmessage = (event) => {
    onMessage(event.data);
  };

  eventSource.onerror = () => {
    onComplete();
    eventSource.close();
  };
}

Step 2: Chat Component Logic

We append the response incrementally:

sendMessage() {
  const question = this.userInput;

  let botMessage = { type: 'bot', text: '' };
  this.messages.push(botMessage);

  this.api.askStreaming(
    question,
    (chunk) => {
      botMessage.text += chunk;
    },
    () => {
      this.loading = false;
    }
  );
}

Rendering Markdown + Code Highlighting

Why This Matters

AI responses often include:

• Code blocks
• Markdown formatting
• Explanations + examples

Without proper rendering → output looks messy.

Step 1: Install Libraries

npm install marked highlight.js

Step 2: Format Function

import { marked } from 'marked';
import hljs from 'highlight.js';
import { DomSanitizer, SafeHtml } from '@angular/platform-browser';

formatMessage(text: string): SafeHtml {

  const html = marked.parse(text) as string;

  const tempDiv = document.createElement('div');
  tempDiv.innerHTML = html;

  tempDiv.querySelectorAll('pre code').forEach((block: any) => {
    hljs.highlightElement(block);
  });

  return this.sanitizer.bypassSecurityTrustHtml(tempDiv.innerHTML);
}

Step 3: Template Rendering

<div *ngFor="let msg of messages">
  <div *ngIf="msg.type === 'user'">
    {{ msg.text }}
  </div>

  <div *ngIf="msg.type === 'bot'" [innerHTML]="formatMessage(msg.text)">
  </div>
</div>

Bonus: Add Theme

In angular.json:

"styles": [
  "node_modules/highlight.js/styles/github.css"
]

Key Challenges and Lessons

1. API Evolution Matters

Initially, I used:

co.generate()

But this has been deprecated.

The correct approach now is:

co.chat() or co.chat_stream()

2. Streaming Requires Different Thinking

• You don’t get full responses
• You handle chunks
• UI must update incrementally

3. Angular + Streaming Needs NgZone

Since EventSource runs outside Angular’s zone:

this.zone.run(() => {
  botMessage.text += chunk;
});

4. Markdown Parsing Is Tricky

• marked.parse() returns string | Promise
• Angular needs string → cast required:

as string

5. Separation of concerns

marked → converts markdown
highlight.js → styles code
Angular → renders HTML

Final Result

At this point, the application supports:

• Real-time AI responses
• Syntax-highlighted code
• Markdown rendering
• Clean Angular UI
• FastAPI streaming backend

This is no longer a demo — it’s a practical GenAI application architecture.

What’s Next?

If you want to extend this further:

✅ Add Chat Memory

Make AI responses context-aware

✅ Add Auto-scroll

Improve chat UX

✅ Add Typing Cursor

Simulate real typing effect

✅ Deploy

• Backend → Render / Railway
• Frontend → Vercel / Netlify

Final Thoughts

Building GenAI apps is not just about calling an API anymore.

It involves:

• streaming
• state management
• rendering pipelines
• user experience

This project gave me a much better understanding of how modern AI applications are built.