Multi-LLM Systems with Abstract Classes in Python

One Interface, Many Models

Blog Thumbnail

Working with more than one large language model in production? Yeah — that gets messy. Different APIs. Varied payloads. Response shapes all over the place. But expectations don’t care. Teams want a simple interface. Something clean they can plug into and not think about what’s happening under the hood.

Which gets us to this: abstract classes. Underrated. Surprisingly powerful. Especially when you’re wrangling multiple LLMs in a unified architecture.

And I’ve been there — trying to shoehorn Claude and GPT responses into a common flow while debugging token mismatches from some half-documented vendor. Not fun. So if you’re building for scale across LLM APIs, here’s a pattern worth adopting early.

The Role of Abstract Classes

Before diving into the LLM stuff, here’s the gist: in Python, abstract classes exist to enforce structure. Think of them like contracts. You define what a subclass must provide without saying how.

Quick glance:

from abc import ABC, abstractmethod

class LLMModel(ABC):
    @abstractmethod
    def call_model(self, prompt: str):
        pass

You’ve got a blueprint. Any subclass of LLMModel must implement call_model(). Can’t skip it. Python won’t allow instantiation otherwise.

That’s good. Because when you’re juggling GPT-4o, Claude 3, Gemini, LLaMA, and Mistral — you need discipline. You can’t have random method variations creeping into prod.

Why You’d Use Abstract Classes with LLMs

Not always necessary, but here’s when they earn their keep:

• You’re wrapping multiple models, and you want one way to call them.
• Your app logic doesn’t care which model it’s using — just that .call_model() works consistently.
• You want plug-and-play support for newer models — drop-in integrations without rearchitecting.

And when not to use them?

Well:

• If you’re working fast and dirty, early prototype-style.
• You’re only using one model. Just hit the API directly.
• You’re not abstracting — the logic is the same regardless.

A Shared Interface Across Popular LLMs

Here’s a rough snapshot of what a multi-model pipeline might deal with:

• OpenAI GPT-4o — expects JSON payloads, completion-style.
• Claude 3 — conversational inputs, system prompts, few-shot examples.
• Google Gemini — different auth model, slightly different field names.
• Meta LLaMA — assuming you’re self-hosting? Format can vary widely.
• Mistral — newer APIs, some idiosyncrasies.

What we want is simple. One interface. One method: call_model(prompt: str). Under the hood, implementation can vary wildly.

Step 1 — Abstract Base

Set your expectations. Force subclasses to implement what matters:

from abc import ABC, abstractmethod

class LLMModel(ABC):
    @abstractmethod
    def call_model(self, prompt: str):
        pass

This defines your surface. Keep it minimal.

Step 2 — Concrete Implementations

Each model knows its API. Knows how to process prompts. Knows what payload the endpoint expects. So we assign that responsibility accordingly:

class OpenAIGPT(LLMModel):
    def call_model(self, prompt: str):
        print(f"[OpenAI GPT] Processing: {prompt}")

class ClaudeModel(LLMModel):
    def call_model(self, prompt: str):
        print(f"[Anthropic Claude] Processing: {prompt}")

class GeminiModel(LLMModel):
    def call_model(self, prompt: str):
        print(f"[Google Gemini] Processing: {prompt}")

class LLaMAModel(LLMModel):
    def call_model(self, prompt: str):
        print(f"[Meta LLaMA] Processing: {prompt}")

class MistralModel(LLMModel):
    def call_model(self, prompt: str):
        print(f"[Mistral] Processing: {prompt}")

Not production-ready. This is scaffolding. Just logs for illustration. In reality? You’d be injecting tokens, forming headers, parsing JSON responses, maybe handling streaming.

Step 3 — Uniform Execution Loop

From a system perspective: run any model, they behave the same:

def execute_model(model: LLMModel, prompt: str):
    model.call_model(prompt)

models = [
    OpenAIGPT(),
    ClaudeModel(),
    GeminiModel(),
    LLaMAModel(),
    MistralModel()
]

for model in models:
    execute_model(model, "Explain the concept of quantum entanglement.")

This — it just works. No conditional logic. No checking if the model is from Vendor A or B.

What You Gain in Production

These pieces start small but pay off over time:

• Extensibility — New vendor tomorrow? Add a .py class, inherit and implement. App code remains untouched.
• Safety — You know .call_model() exists. No KeyError surprises mid-pipeline.
• Isolation — When one model breaks, you don’t drag others down with it.
• Clean logs — Easy traceability. You know where outputs came from.

If teams are swapping between model vendors for latency or cost or performance? This interface holds up. Your orchestration layer stays shallow. Business logic avoids getting tangled up in vendor-specific quirks.

Wrapping It Up

So here’s the takeaway: if you’re architecting around multiple LLM APIs, don’t just write one-off integrations. Abstract early. Not heavily — just enough. Keep your interface narrow and clear.

Even if you only start with two models, you’ll likely add a third. Then another vendor updates their API. Maybe one goes down. Maybe you want fallbacks. That’s when you’ll appreciate the structure you set.

Because what starts out simple can get out of hand — quickly. Abstract classes don’t solve that entirely. But they stop it from dripping into every part of your system.

Further Iterations Worth Considering

• Stream output tokens → useful for time-sensitive applications.
• Retry queues → in case one vendor returns a 429, fallback to another.
• Cache logic → don’t waste tokens if the question has already been asked.
• Multimodal support → maybe audio or image alongside text.

It’s all possible, but first comes structure. Controlled chaos beats uncontrolled chaos every time.

Code is modular. API growth is inevitable.

Plan for both.

Feel free to leave a comment on this blog or reach out to me on

Topmate: https://topmate.io/yash0307jain

Or connect with me on LinkedIn

Linkedin: https://www.linkedin.com/in/yash0307jain/

Thanks for reading, and I’ll see you next time!