Agentic Fullstack Architecture

co-authors: Sagar Sawant

Many folks, even developers, struggle to understand what AI agents are and how then can be used. In IBM’s Chief Data Office (CDO), we have been exploring how to best utilize agents, not just to automate tasks, but to expose to users as much data as possible so that they can self-serve with the end goal of making everyone as productive as possible. This initiative was especially interesting because there was so little data, compared to other efforts we have done with well-established principles, that we had the opportunity to experiment quite a bit. That’s to say, this will showcase one architecture design for a scalable AI implementation that hopefully serves, at a minimum, as a foundation for other applications. A huge benefit of this implementation, compared to others, is that it’s entirely built on open-source tools, so any major limitations are primarily just resources on the machine where it’s run.

End-to-end architecture for a custom user interface leveraging the BeeAI platform and a custom MCP server. [5]

Chat Agents and Large Language Models (LLM)

Chat agents are not new, and IBM certainly made a splash when the Watson DeepQA computer won at Jeopardy! Back in 2011 [1]; but I think its hard to argue that OpenAI’s ChatGPT really kicked off the consumer market when it launched in 2022 [2]. Although all systems are built differently, AI in general has kicked off a wave of productivity and is poised to be the catalyst for massive disruptions in all industries in the foreseeable future; and at the core of an agent are two things: A user interface, the agent, and an LLM. In oversimplified terms, the LLM is a compiled program that has been trained to understand human language and users can interact with it using some form of interface. The first step of this stack is enabling a client interface with an LLM, for which we chose the open-source client BeeAI and the open-source LLM IBM Granite.

Ollama

First, we will need Ollama to provide the IBM Granite LLM

1. Install — brew insall ollama [3]
2. Serve — ollama serve[4]
3. Pull the LLM — ollama pull granite3.3[4]
4. Run the LLM (note that the version of granite may change) — ollama run granite3.3:8b[4]

BeeAI

Next, we will need the BeeAI UI to interact with the IBM Granite provided by Ollama

1. Install uv — curl -LsSf https://astral.sh/uv/install.sh | sh [3]
2. Install BeeAI — *uv tool install beeai-cli *[5]
3. Start the BeeAI platform — *beeai platform start *[5]
4. Configure Ollama — *beeai env setup *[5]
5. *Select LLM provider (type to search): Ollama 💻 local
6. Do you want to use the recommended model ‘granite3.3:8b’? Yes*
7. Launch BeeAI UI — *beeai ui *[5]
8. Open a Browser: localhost:8333 2. Navigate to **chat
9. **Start conversing with the LLM

Following these directions enable you to do the most fundamental thing: interact with the LLM. While this is a big step forward in learning to work with LLMs, its critical to understand what LLMs can and can’t do. LLMs are trained using large data sets in such a way to try to understand human language and produce useful output, like text generation [6]. However, LLMs are limited to the information they are trained on, so to complement gaps in their knowledge one of 3 things can be done:

• Retrain the model — This is generally very expensive
• Create an agent using an LLM and context from a Model Context Protocol server
• Create an agent using an LLM and context from Retrieval-Augmented Generation

Custom Agent with Model Context Protocol (MCP) and Retrieval-Augmented Generation (RAG)

In order to make agents as useful as possible, they need to be integrated with a source from which to pool factual data. The current most common approaches to accomplish this is either connecting to an MCP server or following a RAG implementation. For this stack, we will discuss creating an MCP server; but, for context, the RAF process works by turning data into a vector format, by using something like milvus, that the LLM can easily query and return a formatted answer [7]. MCP servers, on the other hand, expose functionality to agents that is abstracted behind a standardized protocol. This means that agents can query data stores local to the server or the server can query remote servers via API [8].

MCP Server

Our team has been running our APIs for over 7 years, and we found that creating an MCP server from our implementation was very straight-forward: all we had to do was add a few lines of code to enhance our FastAPI APIs with the FastAPI-MCP package. Creating a sample MCP server is quite simple, we’ll create a random number generator from a FastAPI server:

Create a main.py file:

from fastapi import FastAPI
import random

app = FastAPI()

@app.get("/random-number")
def get_random_number():
    return {"random_number": random.randint(1, 100)}

Update the file with FastAPI-MCP [9]:

from fastapi import FastAPI
from fastapi_mcp import FastApiMCP
import random

app = FastAPI()

@app.get("/random-number" , operation_id="generate_random_number")
def get_random_number():
    return {"random_number": random.randint(1, 100)}
mcp = FastApiMCP(
    app,
    name="Random Number Generator ",
    description="API that returns a random number",
)

mcp.mount()

Custom Agent

Key points to remember:

1. The BeeAI Platform is an open-source project hosted by the Linux Foundation. It enables easy discovery, execution, and sharing of AI agents across different frameworks using the Agent Communication Protocol (ACP). It acts as a bridge between diverse agent ecosystems. [5]
2. The BeeAI Framework is a robust open-source framework for building scalable, production-ready multi-agent systems. It supports both Python and TypeScript with full feature parity, offering the performance and flexibility needed for real-world applications. [5]

Depending on your use case, you can create a custom tool using the BeeAI framework. The framework documentation provides helpful guidance, but for a full-stack environment setup, we used sse_client to enable real-time updates via a long-lived HTTP connection. This allows the server to push metadata changes directly to the client. [5]

Using MCP’s async SDK, we open a streaming connection, initialize a session, and load an MCPTool instance, which is then provided to the agent for use. You can choose from any of the agents supported by the BeeAI framework, depending on your needs. We’ve successfully used the ToolCallingAgent and ReActAgent, and also experimented with the RequirementAgent for more complex multi-agent routing setups. [5]

import os
import traceback
import logging
from typing import Any
import httpx

from beeai_framework.backend import ChatModel
from beeai_framework.backend import AssistantMessage, UserMessage
from beeai_framework.errors import FrameworkError
from beeai_framework.agents.tool_calling import ToolCallingAgent
from beeai_framework.agents import AgentExecutionConfig
from beeai_framework.memory import UnconstrainedMemory
from beeai_framework.emitter import EmitterOptions, EventMeta

from acp_sdk import Annotations, MessagePart, Metadata
from acp_sdk.models.platform import PlatformUIAnnotation, PlatformUIType
from acp_sdk.models import Message
from acp_sdk.server import Context, RunYield, RunYieldResume, Server

from beeai_framework.tools.mcp import MCPTool
from mcp.client.session import ClientSession
from mcp.client.sse import sse_client
from collections.abc import AsyncGenerator

from beeai_agents.helper_io import ConsoleReader
from beeai_framework.adapters.beeai_platform.agents import BeeAIPlatformAgent

# ----- Configure Logging -----
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

# ----- Global Initializations -----
memory = UnconstrainedMemory()
reader = ConsoleReader()
server = Server()

# allows the developer to understand the agent thinks patterns based on events
def process_agent_events(data: Any, event: EventMeta) -> None:
    """Log events from agent execution."""
    if event.name == "error":
        reader.write("Agent 🤖 : ", FrameworkError.ensure(data.error).explain())
    elif event.name == "retry":
        reader.write("Agent 🤖 : ", "retrying the action...")
    elif event.name == "update":
        reader.write(f"Agent({data.update.key}) 🤖 : ", data.update.parsed_value)
    elif event.name == "start":
        reader.write("Agent 🤖 : ", "starting new iteration")
    elif event.name == "success":
        reader.write("Agent 🤖 : ", "success")

@server.agent(
    name="ASKME",
    description=("Conversational agent with memory"),
    metadata=Metadata(
        annotations=Annotations(
            beeai_ui=PlatformUIAnnotation(ui_type=PlatformUIType.CHAT)
        ),
        framework="BeeAI",
        recommended_models=["ollama:granite3.3:8b"],
        author={"name": "John Doe"},
    ),
)
async def chat_agent(
    input: list[Message], context: Context
) -> AsyncGenerator[RunYield, RunYieldResume]:
    """Main entrypoint for ACP SDK-based BeeAI chat agent."""

    user_query = ""

    # Extract user query from ACP message parts
    for message in reversed(input or []):
        for part in message.parts or []:
            if part.content and part.content_type == "text/plain":
                user_query = part.content
                break
        if user_query:
            break

    if not user_query:
        yield MessagePart(content="Hello! Please share what you'd like to learn about.")
        return

    logger.info(f"Received user query via ACP SDK: '{user_query}'")

    try:
        server_url = "http://localhost:8080/random-number"
        headers = {}

        # Open SSE connection to MCP
        async with sse_client(
            url=server_url,
            headers=headers,
            httpx_client_factory=create_mcp_http_client_verify_false,
        ) as streams:
            async with ClientSession(streams[0], streams[1]) as session:
                await session.initialize()

                # Load tools registered in MCP
                mcp_tools = await MCPTool.from_client(session)
                for tool in mcp_tools:
                    logger.info(
                        f"🔧 Tool Name: {tool.name}\n📝 Description: {tool.description}"
                    ) 

                # Create and execute the agent
                agent = ToolCallingAgent(
                    llm=ChatModel.from_name("ollama:granite3.3:8b"),
                    tools=mcp_tools,
                    memory=memory,
                )
                response = await agent.run(
                    prompt=user_query,
                    execution=AgentExecutionConfig(
                        max_retries_per_step=3, total_max_retries=10, max_iterations=20
                    ),
                ).on("*", process_agent_events, EmitterOptions(match_nested=False))

                # Store response in memory
                await memory.add_many(
                    [UserMessage(user_query), AssistantMessage(response.answer.text)]
                )

                logger.info(f"BeeAI agent response: {response.answer.text}")

                # sends the response to the user via ACP which the beeai framework will quickly accept
                yield MessagePart(content=response.answer.text)

    except FrameworkError as e:
        logger.error(f"BeeAI framework error: {e.explain()}")
        yield MessagePart(content=f"Error in BeeAI framework: {e.explain()}")

    except Exception as e:
        logger.error(f"Unhandled error: {e}")
        logger.error(traceback.format_exc())
        yield MessagePart(
            content="An unexpected error occurred. Please try again later."
        )

def create_mcp_http_client_verify_false(
    headers: dict[str, str] | None = None,
    timeout: httpx.Timeout | None = None,
    auth: httpx.Auth | None = None,
) -> httpx.AsyncClient:
    """Create an HTTP client that skips SSL verification (use only in dev)."""
    return httpx.AsyncClient(headers=headers, verify=False)

def run():
    """Starts the ACP SDK BeeAI Agent server."""
    logger.info("Starting BeeAI Agent server with ACP SDK...")
    server.run(host=os.getenv("HOST", "127.0.0.1"), port=int(os.getenv("PORT", 8000)))

if __name__ == "__main__":
    run()

Custom User Interface

Now that your agent is set up and registered on the BeeAI Platform, you can leverage the ACP protocol to interact with it using APIs.

The BeeAI Platform supports interaction with your agent via the CLI, Platform UI, and APIs. Once registered, you can use ACP to communicate with the agent programmatically. To do this, you’ll use the OpenAPI specification provided by ACP, which defines how to interact with your agent and should be imported into your preferred API client or SDK. [5]

Below is an example of how you can interact with the Agent using APIs (a small nodejs based code to call the ACP apis which interacts with your agent_name: ‘ASKME’. To maintain session and context you can additionally pass in the session_id to the new runs which will help maintain session for the chats which is always received as a part of the response data during the first run):

const userquery = "Hello I am your query"
   const requestBody = {
      agent_name: 'ASKME', #name of the agent you registered
      input: [
        {
          parts: [
            {
              content: userquery || 'Hello from API',
              content_encoding: 'plain',
              content_type: 'text/plain',
              role: 'user',
              token,
            },
          ],
        },
      ],
      mode: 'stream',
    };

    if (parsedBody.session_id) {
      requestBody.session_id = parsedBody.session_id;
    }

    console.log('----requestBody-----');
    console.log(requestBody);

    try {
      const response = await axios({
        method: 'post',
        url: 'http://127.0.0.1:8333/api/v1/acp/runs', #platfrom url if running on local
        headers: apiheaders,
        data: requestBody,
        responseType: 'stream',
      });

      const result = {
        session_id: null,
        run_id: null,
        status: null,
        messages: [],
        raw_events: [],
      };

      let buffer = '';

      await new Promise((resolve, reject) => {
        response.data.on('data', (chunk) => {
          buffer += chunk.toString();

          const lines = buffer.split('\n');

          for (let i = 0; i < lines.length; i++) {
            const line = lines[i].trim();
            if (!line.startsWith('data:')) continue;

            const jsonString = line.replace(/^data:\s*/, '');

            if (jsonString === '[DONE]') {
              resolve();
              return;
            }

            try {
              const parsed = JSON.parse(jsonString);
              result.raw_events.push(parsed);

              switch (parsed.type) {
                case 'run.created':
                case 'run.in-progress':
                case 'run.completed':
                  result.run_id = parsed.run?.run_id ?? result.run_id;
                  result.session_id = parsed.run?.session_id ?? result.session_id;
                  result.status = parsed.run?.status ?? result.status;
                  break;
                case 'message.part':
                  if (parsed.part?.content) {
                    result.messages.push(parsed.part.content);
                  }
                  break;
                case 'message.completed':
                  if (parsed.message?.parts) {
                    parsed.message.parts.forEach((p) => {
                      if (p.content) result.messages.push(p.content);
                    });
                  }
                  break;
              }
            } catch (err) {
              log.error(`Malformed JSON: ${jsonString}`);
            }
          }

          buffer = lines[lines.length - 1]; // preserve partial data
        });

        response.data.on('end', () => resolve());
        response.data.on('error', (err) => reject(err));
      });

      return res.status(200).send({ data: result });
    } catch (error) {
      if (error.code === 'ECONNABORTED') {
        log.error('Request Timeout in ASKME');
        return res.status(500).send({ error: 'Request timed out' });
      }
      log.error(`Error in ASKME: ${error.message}`);
      return res.status(500).send({ error: error.message });
    }

Sources​​​

[1] “Watson, ‘Jeopardy!’ champion,” [Online]. Available: https://www.ibm.com/history/watson-jeopardy. [Accessed 1 July 2025].

[2] K. Wiggers, C. Corrall, A. Stringer and K. Park, “ChatGPT: Everything you need to know about the AI-powered chatbot,” TechCrunch Media LLC, 30 June 2025. [Online]. Available: https://techcrunch.com/2025/06/30/chatgpt-everything-to-know-about-the-ai-chatbot/. [Accessed 1 July 2025].

[3] “Homebrew Formulae — ollama,” [Online]. Available: https://formulae.brew.sh/formula/ollama. [Accessed 1 July 2025].

[4] “Github — Ollama,” [Online]. Available: https://github.com/ollama/ollama. [Accessed 1 July 2025].

[5] “BeeAI,” IBM, [Online]. Available: https://docs.beeai.dev/introduction/welcome. [Accessed 1 July 2025].

[6] “What are large language models (LLMs)?,” IBM, 2 November 2023. [Online]. Available: https://www.ibm.com/think/topics/large-language-models. [Accessed 1 July 2025].

[7] “Retrieval-augmented generation (RAG) pattern,” IBM, 1 May 2025. [Online]. Available: https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-rag.html?context=wx. [Accessed 1 July 2025].

[8] “Introduction,” Anthropic, PBC, [Online]. Available: https://modelcontextprotocol.io/introduction. [Accessed 1 July 2025].

[9] R. “Integrating MCP Servers with FastAPI,” 12 May 2025. [Online]. Available: https://medium.com/@ruchi.awasthi63/integrating-mcp-servers-with-fastapi-2c6d0c9a4749. [Accessed 1 July 2025].

Disclaimer

The above article is personal and does not necessarily represent IBM’s positions, strategies, or opinions.​