Building an MCP-Based AI Network Engineer with Python

I recently built a small AI-powered network assistant using Python, Groq LLMs, FastMCP, and Netmiko to understand how modern LLM workflows can interact with real network devices.

The idea was simple:

Ask networking questions in plain English → have an LLM decide what network data is needed → connect to a router → retrieve live operational state → let the LLM analyze the output and respond naturally.

The interesting part was not just the LLM itself, but understanding how all the components communicate internally.

High-Level Architecture

The workflow looks like this:

User
 ↓
Groq LLM
 ↓
Tool Selection Logic
 ↓
FastMCP Tool
 ↓
Netmiko
 ↓
Cisco Router
 ↓
CLI Output
 ↓
Groq LLM Analysis
 ↓
Final Human Response

The project is split into two logical parts:

1. LLM orchestration (main.py)
2. Network tools exposed through FastMCP (tools.py)

Step 1: Building the Router Connectivity Layer

For device communication, I used Netmiko.

The actual communication path here is:

Python
 ↓
Netmiko
 ↓
Telnet/SSH Session
 ↓
Cisco CLI

Netmiko internally handles:

• session establishment
• prompt detection
• command execution
• output collection

Example tool:

@mcp.tool()
def show_interfaces():
    conn = ConnectHandler(**router)
    output = conn.send_command(
        "show ip interface brief"
    )
    conn.disconnect()
    return output

This tool simply:

1. Opens a session
2. Executes a CLI command
3. Collects output
4. Returns the result as text

Step 2 : Why FastMCP Was Added

I could have directly called Python functions. But I wanted to structure the project around MCP (Model Context Protocol). FastMCP exposes Python functions as discoverable tools for AI systems.

Instead of manually wiring every function:

if "bgp" in prompt:
    show_bgp()

FastMCP allows tools to become standardized callable capabilities.

Example:

@mcp.tool()
def show_bgp():

Now the function becomes:

• discoverable
• reusable
• AI-friendly

Conceptually:

Without MCP:
LLM → custom Python logic
With MCP:
LLM → standardized tools layer

This becomes important as systems scale:

• multiple devices
• telemetry sources
• ticketing systems
• observability platforms

Step 3: LLM Integration with Groq

For inference, I used Groq with the openai/gpt-oss-120b model.

The helper function is intentionally simple:

def ask_llm(messages):
    completion = client.chat.completions.create(
        model="openai/gpt-oss-120b",
        messages=messages,
        temperature=0
    )
   return completion.choices[0].message.content

At this stage, the LLM performs two different jobs:

1. Tool selection
2. Output analysis

These are actually separate reasoning stages.

Step 4: Tool Selection Stage

The first prompt determines:

Which network tool should be executed?

The system prompt constrains the model:

SYSTEM_PROMPT = """
You are a network automation assistant.
If the user asks about interfaces:
return ONLY:
TOOL:show_interfaces
"""

Example flow:

User:

give me status of all interfaces

LLM response:

TOOL:show_interfaces

At this point:

• No network data has been analysed yet
• The model is only deciding intent

This is essentially an AI-driven command dispatcher.

Step 5: Network Device Execution

Once the tool is selected:

tool_output = show_interfaces()
Netmiko connects to the router and executes:
show ip interface brief

The raw CLI output is returned to Python.

Example:

Ethernet1/0 up up
Ethernet1/1 up up
Ethernet2/0 administratively down down

At this stage, the system now has a live operational network state.

Step 6: Second LLM Pass

Initially, my implementation stopped after returning raw CLI output.

That worked for:

• “show interfaces”
• “show bgp”

But failed for analytical questions like:

• “How many interfaces are up?”
• “Which neighbors are down?”

The reason was:

• the LLM selected tools
• but never interpreted the returned data

The fix was introducing a second LLM reasoning stage.

The raw router output is fed back into the model:

analysis_messages = [
    {
        "role": "user",
        "content": f"""
User Question:
{user_input}
Router Output:
{tool_output}
"""
    }
]

Now the LLM performs actual analysis.

Example:

User:

How many interfaces are up?

Router Output:

Ethernet1/0 up up
Ethernet1/1 up up
Ethernet2/0 administratively down down

Final AI Response:

2 interfaces are operationally up.
1 interface is administratively down.

This is where the assistant becomes genuinely useful.