Build, buy, or train? An AI reality check for SMEs

Our guest spent years building custom AI for regulated industries. His advice for smaller businesses is refreshingly un-hyped: most of you shouldn’t be training anything.

[embed]

There’s a seductive idea doing the rounds right now: AI has made it cheap and easy to build your own software, your own models, your own everything. Prompt away on Lovable, wire up an agent, and who needs vendors any more?

Our third guest, Mark Johnston, has actually done the hard version of this. He cut his teeth in 90s telecoms and the US dot-com boom, ran educational services and a machine-learning research centre at Cambridge, and went on to co-found — and exit — a regulatory-compliance company with custom AI at its core. So when someone who has trained domain-specific models for a living tells you to slow down, it’s worth listening.

TLDR: Most SMEs don’t need to train a model. Learn the difference between retrieving knowledge (building a RAG) and generating a proprietary outcome (maybe training). Buy-versus-build hasn’t gone away — the boundary has just moved, and vendor lock-in now comes with a variable token bill. AI has made it easier than ever to build the wrong thing quickly, so get the fundamentals right first. And remember: your real advantage as an SME isn’t budget — it’s how fast you can move.

A football stadium versus a room of lawyers

Mark’s favourite analogy is worth stealing. Asking a large language model a question, he says, is like walking into a football stadium and shouting it at the crowd. A few people in there have the relevant expertise and will give you a brilliant answer. Most will give you something general, which is rather the point of a general model.

A domain-specific small model is different. It’s like walking into a room with fifteen lawyers and getting a legal opinion. Train a small, open-source model on your own proprietary data, and you get the behaviour you actually want: higher quality, guardrails you can rely on, outputs you can quality-assure — and, because it’s smaller, something that’s faster and cheaper to run.

That sounds appealing. It’s also where most of the nonsense starts.

Takeaways for SMEs

1. Be honest about whether you need to train anything

Training your own model is a genuinely hard technical job. It’s not just the data science — the model has to live somewhere, be fed, monitored and maintained, which means compute, storage, pipelines, software engineering and infrastructure. If you’re a non-technical SME, you don’t have that bench.

So before anyone reaches for it, ask what you’re actually trying to do. If you just need to retrieve information, you don’t need to train a model — put a RAG layer over an LLM and let it pull on your data. You’d only train a model if you have defensible, proprietary data and you want it to produce a proprietary outcome your competitors can’t replicate. No unique data, no unique outcome? It’s almost certainly not worth it. And whatever you do, don’t hand your crown-jewels data to a public LLM — keep it private.

2. Buy-versus-build didn’t disappear — the boundary moved

The fashionable line is that AI killed the buy-versus-build question. It didn’t; it just shifted where the line sits. SMEs have historically bought software for good reasons — it’s risky and expensive to build, and you’re rarely buying “software” anyway. You’re buying an outcome wrapped in reliability, support, security, compliance and maintenance.

Vibe-coding your way to a bespoke platform doesn’t make those needs vanish. It usually swaps a predictable, fixed software cost for a variable token or compute bill — and ties you into whichever low-code ecosystem you built it in. Switching software vendors is already painful; extracting yourself from a proprietary platform you built is harder still. Worth remembering, too, that today’s token prices are heavily subsidised. Those costs are far more likely to rise than fall.

3. Add intelligence on top — don’t rip and replace

Ignore the “SaaS apocalypse.” You don’t need to throw out your stack and move to untested “AI-native” everything. Far more sensible for most SMEs: keep your existing systems and add an intelligence layer on top of them. An agent that pulls insight out of your CRM. A wrapper that joins up the fragmented data trapped across your spreadsheets and turns it into something useful. Best of both worlds, and a fraction of the risk.

4. It’s never been easier to build the wrong thing

This was Mark’s sharpest point, and it earned a “put it on a t-shirt” from Theo: AI has lowered costs so far that it now lets anyone build the wrong thing faster and cheaper than ever. Which makes the boring fundamentals more important, not less. What problem are you actually solving? Rubbish in still gets you rubbish out, no matter how much AI sits in the middle. Plenty of the real wins turn out to be people, process and a decent data clean-up — not a model at all.

Where SMEs actually win

The temptation is to treat your size as a disadvantage. It isn’t. AI compresses the research-and-development cycle — you can research, run experiments and stand up prototypes with one or two people — work that used to require a whole R&D function. Most SMEs can do far more exploratory work than they realise.

And big businesses are supertankers: they instinctively protect the existing business from anything that might disrupt it, and they turn slowly. SMEs are nimble and interconnected, carry more of the knowledge in fewer heads, and can join the dots and move. The interesting question isn’t “how do I bolt AI onto what I already do?” It’s “what would an AI-enabled version of this business actually look like?” — and you’re far better placed to answer that than the supertanker is.

What should you do?

1. Diagnose before you build. Decide whether you need retrieval (RAG), a custom model, or — often — neither. Don’t train anything unless you have genuinely defensible data and a proprietary outcome to chase.
2. Cost the whole thing, not the licence. Map how token and compute costs scale, and assume today’s subsidised prices won’t last.
3. Layer, don’t lurch. Add intelligence to the systems you already trust before you consider replacing them.
4. Fix the fundamentals first. Get clear on the problem and sort the people, process and data before AI touches any of it.
5. Use your speed. Treat AI as a cheap R&D function and ask what an AI-enabled version of your business could become.

Conferences and demos make all of this look effortless. The real work is the unglamorous bit: deciding what’s worth building in the first place, and having the nerve to call the rest of it what it is.

We can help with that.

This conversation is from Episode 16 of Beside Ourselves, the Beside Partners podcast on AI in business, with guest Mark Johnston. Listen on Spotify, Apple Podcasts, or YouTube.