Why I Am Anxiously Waiting For the Mac mini M5 to Build My Local AI Box: The Math Behind It

The mini was absent at WWDC in June. The M5 silicon shipped already. Heres the spreadsheet I created before I decided to put my wallet away a little longer.

Eight days ago I wrote a post about how I finally found a local coding model worth holding on to. Well — Qwen 3.6–27B stuck (and that says a lot, after years of trashing every local LLM I tried within a fortnight). Which is takes on, like 17GB of my 36GB MacBook Pro running this successfully and chewing through real refactors and a this really ugly SQL bug from the backlog I had to fix with it still installed I used it this morning. That article concluded with a sort of settled, jolly note. This one is about the scratch it left behind.

Its because there is a difference between a machine where you can run the model and one that you actually want to be running it. My laptop is perfectly adequate on paper; the 27B fits with room to spare. You get into trouble the instant this model is resident with a 32K context and everything else that fits in between eight hours of my working day, a browser running forty different tabs, a simulator, a dev server. Is when the unified memory gets tight, the machine starts swapping, and everything suddenly becomes slow. Not that the Mac is incapable of running the model, rather it’s just that the Mac happens to also be the computer I want to work on while it runs the model. I am out there fighting with the model over fresh RAM — and losing. That was what led me to seek out a second box, one that solely holds a model and replies if queried.

A few weeks back I wrote about NVIDIA’s Spark and got really excited by it, still stand by that: it’s the most powerful local AI box you can put on a desk at the moment — with 128GB unified memory & headroom for models up to ~200B parameters. But, it is also an intentional, extravagant, energy-guzzling apparatus for individuals whose models actually require that kind of space. The longer I sat with my own habits, the more my gaze returned to the most boring computer Apple sells. A Mac mini sitting there on a shelf, sipping power and quietly responding to requests. The spark is at best the machine you buy when you have outgrown everything else; the mini the one for which you can be truthful that you haven’t. That second machine is the form factor of the device I am actually looking for, and the caveat being that I want one in a variation not available yet: The M5 mini.

Why a separate box at all

The real reason I want a separate box is nothing to do with raw speed. Pass data passing, it is all about sepode concerns. That forever-on machine doing just one thing leaves my model resident in memory which means I can point my editor and a couple of small local network tools at it, and never again stress over watching portions of this laptop locking up trying to both be the inference server and be the thing that I am working on.

Apple’s mini seems almost suspiciously perfect for that job. It is compact, it idles quietly under loads that would cause a laptop fan to scream, and (the part that’s relevant for this entire exercise) it has unified memory where the CPU, the graphics cores, and the Neural Engine all share one pool of identical memory. Because there is no separate graphics card, you do not spend time copying tensors across a slow bus to the graphics card. That architecture is exactly what we want for large-model inference and why a fanless-feeling appliance keeps popping into these conversations next to machines that pull 10x as many watts.

Hence the shape of the mini is correct. The problem is the moment you try to spec one, because the first and most important number is never a number you can change.

It was never about the chip. It is also about memory you can not upgrade later

With Apple silicon, the unified memory is soldered in and configurable only once, ever. There is no, you know, cracking that case-sized thing open on your first year and dumping more in. Anything you purchase on day one is as good as it will ever get for everything you will ever load on that machine for the rest of its life. So the number matters enormously. What I took a while to realize is which way.

Last week I wrote about the following lesson that I learned the hard way (I guess you pay for some lessons right): knowing if a model fits on any Mac is not the same as whether a model runs. Fitting is a memory-capacity problem. All the running fast works is a memory-bandwidth problem, every single token has to drag the active weights across via memory in a big dense 70B model and you slap that onto a mac with enough RAM and it will quite happily load then generate maybe four tokens per second slower than I can type. Thus “buy the most memory and run the biggest model” is exactly the trap. The models that actually deserve to earn their stripes are the fit and able: mid-size dense models in the 27-to-32B parameter range, or mixture-of experts (MoE) models that have a ton of knowledge but only fire a handful of billion parameters per token.

That reshapes what the memory is even of. The base M4 mini has a floor of 16GB and offers up to about 32GB conformed, while the M4 Pro has an entry-level spec of 24GB with a cap of 64G. That’s also the ceiling for practically all present day min line plans since it’s in-soldered and finished components as factual design constraints. There are however some options, I do not want that 64GB so I can cram in a 70B model I’m abandoning at four tokens a second. I want it so that I can run a fast 27B resident with a huge context window and still have headroom to do everything else the box is doing without any swapping that makes my laptop miserable today. I am paying for breathing space, not a bigger number on the model card

The question I am really asking is not “how big of a model can this thing load. It is “would it allow the model I actually use to be speedy and resident, with headroom for years, even if I knew that I could never add one more gigabyte ever.” That choice is the one permanent decision — everything else gyrates around it.

What M5 actually gets you (what it does not)

Now, the chip is not nothing. This is the moment when waiting becomes less stalling and more math.

The M5 already exists. This isn’t speculation about a ghost, since Apple fitted it into the 14-inch MacBook Pro in November. We have real numbers. In a nutshell for anyone running models locally, the M5 incorporates dedicated Neural Accelerators within its graphics cores purpose-built to service the matrix multiplication at the core of every transformer. They were integrated into Apple’s proprietary MLX framework, and the gap is apparent even based simply on benchmarks: between about 19 to 27 percent improvement across language-model workloads versus the M4; in fact larger M5 chips (even at identical clocks) can squeeze as much as roughly seven additional tokens per second per clock cycle out of their grid with potentially up to a ~28 percent real-world advantage over their M4 equivalents.

But the number that is truly a day changer for me, though, is time-to-first-token — the delay between hitting enter and the model saying something. For a dense 14B model on an M5 MacBook Pro, that translates to a delay of less than 10 seconds which is reduced to ❤s for a mixture-of-experts with 30 billion parameters. And that is the difference between a tool you instinctively reach for, and one you avoid — because using it feels as if you’re filing requests into a government office with litigious latency.

Memory bandwidth moved too. The vanilla M5 browses approximately 153 GB/s over the M4’s 120 — around 28 percent more pavement for the data to run on — and bigger variants go very higher. When it comes to inference, raw compute is often not the bottleneck; bandwidth often is — making lift less like a footnote.

What M5 does not do, at least on the historically built mini, is shift that 64GB ceiling. It fast forwards how quickly a chip can suck in 32B model to generate tokens It doesn’t stretch a 70B model to where it physically couldn’t fit in earlier times. If your genuine workload lives at or below that 30B threshold, as mine does, the M5 is a quantifiable step forward. Your new silicon will be useless inside this chassis if your aspirations live above it. Well, that is the very gap filled by that NVIDIA Spark (128GB, 200B model fit) I wrote about and pays for a rent-an-hour of cloud compute by long. The mini is not that machine, and pretending it is will land you a 64GB box gagging on the kind of model that was never going to fit. Choose the right tool for the job I’ll come back to that.

The boring math: Owning a 5-watt box vs renting an H100

The answer quieting the “own versus rent” debate in my brain, this is less self-evident than hardware forums suggest it is.

Cloud GPUs are appealing because the hourly rates seem nearly free. You can actually score an H100 on a spot marketplace for between one dollar and a dollar-forty an hour. If heavy hitters for an afternoon set you back the price of a sandwich, well, those boxy bills would cost five figures to own. Best would be renting for bursty, occasional and truly heavy training-shaped work (and I wont argue against that).

However, that is not the kind of workload I have and I imagine it is the same for most people. My usage is a trickle: a query here, a small retrieval job there, a (long-running) assistant that mostly just sits around and sometimes ponders. Cloud billing penalises exactly that behaviour, since you are billed for the wall-clock time the instance is alive, not just the seconds it spends computing. If you have an all-day accessible assistant, that means paying the entire day. Its one small trickle on the spreadsheet which turns into a flood on the invoice.

The case of the mini is an inverse of that. The idle of a mini idles at around three watts and peaks at about forty-five under load. The cost of electricity when left running constantly at idle comes in at less than five bucks a year. Not a typo. A year. Now compare that to a discrete desktop GPU such as a 4090 drawing more than five hundred watts easily able to silently bleed sixty-odd bucks each month from your power bill just keeping the lights on. But cheap to leave on is exactly the what always available local model has to be, and it is all of first gen mini’s proposition for value.

So the real comparison is NOT mini vs rented H100 on raw performance, because the mini loses that one on paper and I don’t care about it. It is a one-off hardware cost plus an electricity bill that as far as I am aware rarely goes beyond rounding error vs a meter that goes on and on endlessly for however long you want the thing reachable. The boring little box wins, easily, with two cumulative years of consistent but low-impact usage.

I want to be careful here because I made this point in the coding piece too and am not going to contradict myself. The box does not become the cloud. For the toughest issues, where that bug is really just a false assumption blasted across a codebase, I still escalate to a paid frontier model as I always have. In my case, zero dollars did not continue to flow into the pockets of my API; at least it was cut in half. The argument isn’t “cancel your subscriptions”. That is: all of the predictable, unsexy bulk of work never actually required a frontier model in the first place, and borrowing one by the hour for it us where everything stays quiet but burns cash.

So why am I actually waiting?

With all that, the simple question is why don’t I just buy a 64GB M4 Pro mini today and leave it at that?

There are reasons — well, a couple of them are more stubborn than logical.

The first is really about that irreversible memory decision. When you are buying a machine that has memory I can never upgrade, and when a new generation is clearly right around the corner with calculating fibres proven to be factually superior bandwidth along with an actual Neural Engine built for this work, spending serious coin on last seasons chip the very season before its replacement feels like small, self-inflicted nick. It’s not the end of the world, but something I would quietly regret shifting away from in my head every time a chip benchmark came out for a part I didn’t wait 3 months for.

The second is I’m waiting to see the M5 mini’s memory tiers and price before I make a decision, as both of those are still question marks. The previous low entry price was killed off by Apple so the next mini is apparently set to start a little higher, in the seven hundreds as opposed to the old five-ninety-nine. Even more crucially, I want to know if the M5 Pro mini raises the ceiling beyond 64GB or holds firm, because that one item is what determines if potentially the largest options I am interested in ever comes within range of this form factor.

And the third of which I keep arguing with myself. The mini could in fact be pushed into the second half of this year or even later. Absent from WWDC, Apple is grappling with supply and the same AI-driven demand crunch that everyone else is. You end up waiting forever because you are waiting for a product that is not on the books. Eventually, it just becomes cheaper to simply buy the somewhat-older one than trying to avoid buying that box. I am not at that point. But I could be by then, and if an M5 mini is still absent from the lineup in autumn, I’ll probably swipe my M4 and take it out with me.

That’s where I’m at, and it’s not a nice place to land: there’s the math that says wait, this calendar that says you probably shouldn’t be waiting, and then there’s the number which dictates the fate of my purchase that’s completely unseen until Apple comes clean. Therefore, I sit on my hands until the launch that is near but has a hardstop around three months out.

If you are considering the same purchase, please do not interpret “buy a mini” or “wait for M5” to be the moral of this story. It is the model that you actually reach for at least on a day-to-day basis, the fast version that stays in your head before you think about it to use; so purchase memory sufficient enough to keep it available with room to stretch years out. I have purged enough of these to learn: Do not purchase memory for a larger model that you will throw out at four tokens per second. A chip that runs faster is something you can afford to dismiss. This memory that you are stuck with, that’s the part you have to deal with.

Continue reading

**I Went Looking for the Best Local LLM for Coding in 2026** — the model (and the 36GB reality) this whole post grows out of.