SubQ: What Actually Changed (And What’s Vendor-Run)

A Miami startup says it built the first frontier non-transformer LLM with a 12-million-token context and 1,000x efficiency gains. The CTO later confirmed the model builds on someone else’s open-source weights. Here’s the architecture, the benchmarks, the caveats — and the part the launch post left out.

On May 5, 2026, a Miami-based startup called Subquadratic came out of stealth with $29M in seed funding, a thirteen-person team, and a single, very loud claim: they had built the first frontier large language model to escape quadratic attention. The model is called SubQ. The advertised context window is twelve million tokens. The headline number, repeated across launch coverage, is a 1,000x compute reduction over dense attention at long context.

The AI engineering community split within hours. Either this is the most significant architectural shift since the 2017 transformer paper, or it joins a long line of subquadratic experiments that worked beautifully in a paper and disappointingly in production. Neither read is a useful starting point for a practitioner trying to decide whether to care.

Three weeks later, the picture is clearer. Two things about the launch are genuinely interesting. Three are overstated. And one admission made by the CTO in the hours after launch — in response to a critic on X — reframes how to read the entire pitch.

Here’s what changed, what didn’t, what was claimed, and what was quietly conceded.

The Quadratic Ceiling — And Everything That’s Already Tried to Break It

If you don’t work in ML, here’s the version that matters. In every transformer-based model — ChatGPT, Claude, Gemini, all of them — each token in your prompt is compared against every other token in your prompt. Double the input length and you don’t double the work, you roughly quadruple it. This is the “quadratic attention” problem. It’s the reason long-context AI is expensive, the reason retrieval-augmented generation (RAG) became a default architecture pattern, and the reason most production systems chunk inputs rather than process them whole.

The research community has been trying to escape this for years. State-space models like Mamba and RWKV replace attention with recurrent state that scales linearly. Hyena, RetNet, and BASED took different swings at the same trade-off. Longformer and BigBird used fixed-pattern sparse attention — choosing which tokens attend to which by position, not content. DeepSeek’s recent Sparse Attention work and Kimi Linear each shipped variants of these ideas at frontier labs.

The pattern across this lineage is consistent. Each approach achieves the linear scaling it claims. Each one then either (a) underperforms dense attention on downstream benchmarks at frontier scale, or (b) survives by becoming a hybrid that retains dense attention layers — which means the quadratic cost reappears in the layers that matter most. A widely circulated LessWrong analysis from January 2026 argued that nearly all subquadratic claims to date are best understood as “incremental improvement number 93595 to the transformer architecture” — constant-factor speedups, not architectural shifts.

That is the field SubQ is launching into. It is not an empty field. It is a field paved with serious attempts that didn’t displace dense attention. That context matters before reading a single benchmark.

What Actually Shipped on May 5

SubQ is built around a mechanism Subquadratic calls SSA — Subquadratic Sparse Attention.

In plain terms: instead of comparing every token against every other token, SSA lets the model learn, for each query, which positions in the sequence are worth looking at, and computes attention only over that subset. The selection is content-dependent (the model picks based on what each token means), not position-based (like Longformer’s fixed windows). It is not a state-space model — attention is still the core operation, but the model chooses where to attend rather than attending everywhere.

Technically, this gives SSA three properties that have rarely come together: linear scaling in compute and memory, content-aware routing, and exact retrieval from arbitrary positions in the sequence. Prior architectures have generally landed two of those three. If SSA delivers all three at scale, it is a meaningful contribution.

The launch shipped three products, all private beta:

• SubQ API — an OpenAI-compatible endpoint exposing a 1M-token production model. The 12M window is research-only, gated to select partners.
• SubQ Code — a CLI coding agent that loads whole codebases into context.
• SubQ Search — a free long-context research tool, positioned against Perplexity and ChatGPT search.

Subquadratic published three benchmark results: 95.6% on RULER at 128K tokens (vs Claude Opus 4.6’s 94.8%); 65.9% (production) or 83 (research) on MRCR v2 at 1M tokens (vs GPT-5.5’s 74 and a confusing pair of Opus numbers we’ll come back to); and 81.8% on SWE-Bench Verified (vs Opus 4.6’s 80.8%). On the wall-clock side, they report 52.2x faster prefill than FlashAttention-2 at 1M tokens on Nvidia B200s. Those wall-clock and FLOP-reduction numbers were reviewed by Appen at the kernel level — the only externally verified piece of the pitch.

Then, within hours of launch, the conversation changed.

The Admission That Reframes the Pitch

The launch page describes SubQ as “a ground-up redesign of how attention works, built to be subquadratic from first principles.” The launch coverage in SiliconANGLE, eWeek, The New Stack, and elsewhere repeated that framing. The implication: a new architecture, trained from scratch, breaking the transformer monopoly.

Within hours, former OpenAI researcher Will Depue posted that SubQ was “almost surely a sparse attention finetune of Kimi or DeepSeek” — meaning the base model behavior comes from existing open-source weights, with the SSA mechanism added on top. CTO Alex Whedon replied on X, confirming the point. The company is, in Whedon’s words, “using weights from open-source models as a starting point, as a function of our funding and maturity as a company.” Coverage at DataCamp identified the most likely base as a member of the DeepSeek V4 family.

That admission is not a scandal. Starting from open weights is a perfectly defensible engineering choice for a 13-person company with $29M to spend. It is, however, a very different story than “first frontier model built from first principles to be subquadratic.” A sparse-attention layer grafted onto somebody else’s pretrained model is a legitimate contribution. It is not the same contribution.

Depue followed up arguing that the company’s O(n) scaling claims and the reported speedup numbers “don’t seem to line up,” calling the communication “either incredibly poorly communicated or just not real.” Subquadratic responded with the technical post detailing the SSA mechanism. That post is informative and worth reading on its own merits — and credit to Whedon for engaging publicly within hours. But it does not include a full model card, a peer-reviewed paper, or weights.

What SubQ Solves — If the Numbers Hold

The interesting target SubQ is aiming at is not nominal context length. Every frontier lab in 2026 advertises a 1M-token window. Almost none of them reliably reason over the full window. The gap between nominal context (what the model accepts) and functional context (what the model can actually use) is one of the most important unsolved problems in production AI, and the reason RAG, chunking, and orchestration scaffolding have proliferated.

If SSA delivers what Subquadratic claims, the practical effect is this: a much wider set of long-context workloads becomes economically viable. Codebase-scale coding agents that fit an entire repository into one call. Enterprise document review that holds an entire contract corpus in view. Legal discovery without retrieval pipelines. Multi-day agentic sessions that preserve full prior state. These are not abstractions — they are workloads currently blocked by per-token cost at long context.

That conditional matters. The numbers driving the pitch — the 1,000x compute reduction at 12M tokens, the 300x cost gap on RULER 128K — are at scales where independent verification is hardest to come by. Which brings us to the part of this launch that needs to be read carefully.

The Limits and the Hype Check

This section is where any honest read of SubQ has to live. Several things are quietly going on at once.

No technical report, no open weights, no arXiv. Three weeks in, none of these exist. The “model card coming soon” line has been on the technical post since May 5 with one update on May 15 referencing Appen’s kernel review. The accuracy benchmarks — RULER, MRCR v2, SWE-Bench — were not independently re-run. There are no leaderboard entries on LMArena, Artificial Analysis, or LiveBench. By the standards of how the field evaluates frontier claims, the verification surface is roughly empty.

Single-run benchmarks. Per The New Stack, each benchmark was run only once due to high inference cost, with no confidence intervals reported. The SWE-Bench delta over Opus 4.6 (81.8 vs 80.8) is, in the company’s own framing, “harness as much as model” — meaning the margin is comfortably within run-to-run variance. Whedon himself described SubQ as “way smaller than the big labs,” which is a fair acknowledgment but also a reason single-run results need replication before they’re load-bearing.

The MRCR v2 problem. Subquadratic reports two MRCR v2 numbers for SubQ: 83 for the research model, 65.9 for the production model. That is a seventeen-point gap on the same benchmark for the same model family. Independent verification analyst Jake Cuth noted that the gap is larger than the spread between most frontier models’ end-to-end results. The production number — the one a developer would actually hit — trails GPT-5.5’s 74.0%.

Comparator drift. The launch press cites Claude Opus 4.6 numbers (78.3 on MRCR v2). The technical post on Subquadratic’s own site cites both 4.6 (78.3) and 4.7 (32.2). Opus 4.6 outscores SubQ’s production number on MRCR v2 (78.3 vs 65.9); Opus 4.7 has apparently regressed on that benchmark to 32.2, and SubQ beats it there. Which comparator a reader gets depends entirely on where they land in Subquadratic’s communications. That kind of version-shopping is one of the most common patterns of benchmark cherry-picking.

The base-model admission. Once Whedon confirmed that SubQ builds on open-source weights, the claim “first frontier non-transformer LLM” becomes harder to defend. SubQ is a transformer-derived model with attention layers replaced by SSA, trained further. That’s interesting; that’s just not what the launch coverage described.

Cost claims cannot be verified. Subquadratic told SiliconANGLE that SubQ hits 95% on RULER 128K at $8 of compute against ~$2,600 for Opus to hit 94% — a 300x cost reduction at parity accuracy. The company has not published API pricing. The cost claim therefore cannot be independently checked.

Magic.dev parallel. In August 2024, Magic.dev announced a 100-million-token context model with a claimed 1,000x efficiency advantage and went on to raise more than $500M on the strength of those numbers. Nearly two years later, there is no public evidence of LTM-2-mini in production use outside Magic. Subquadratic’s seed valuation is reported at around $500M, with no public weights, no peer-reviewed paper, and a 12M-token claim resting on self-reported single runs. The structural similarity is uncomfortable. It is not an accusation. It is the most relevant base rate.

None of this means SubQ is fake. The architecture description is technically coherent. The Appen-verified kernel speedups are real. Sparse content-dependent attention is a serious research direction. What it means is that the gap between the launch framing and the verifiable evidence is wider than the launch framing acknowledges.

Who Should Care, and When

For end users: Nothing to do today. SubQ is private beta. Whatever you use — ChatGPT, Claude, Gemini — your experience does not change this week.

For API developers: Do not migrate based on the launch numbers. The 1M-token production model is OpenAI-compatible, but you cannot evaluate cost without published pricing, and you cannot evaluate quality without independent reproduction. If you have a workload that is genuinely long-context bound — a coding agent that needs whole-repo context, a document review system stuck on chunking errors, an agentic session that fails on context boundaries — request early access and run your own evals on data you understand well. Compare to Gemini 3.5 Flash (1M context, GA, $1.50/$9 per million tokens), Claude Opus 4.7 (1M context), and your current stack. The fair test is your data, not their benchmarks.

For ML practitioners and researchers: The SSA description is interesting on its own — particularly the critique of DeepSeek Sparse Attention’s quadratic indexer. Read the technical post. Wait for the model card. Treat the architecture claim as a hypothesis until weights or a paper land.

For self-hosters: Not applicable. Weights are not open, and the company has signaled no near-term plan to release them.

For decision-makers betting roadmap on long context: The honest read is that SubQ shifts the expected value of long-context architectures becoming dramatically cheaper, but not by enough to commit production roadmap to. If you’ve been waiting for genuine functional-context capability before building a class of products, SubQ is a signal worth tracking. It is not a signal to ship against.

What I’m Watching

A few specific things over the next quarter will resolve which read of SubQ is correct.

First, the model card and technical report. Subquadratic has said both are coming. The contents — particularly the training methodology, evaluation protocol, and any reported failure modes — will tell us whether the architecture is what the company says it is.

Second, leaderboard entries. LMArena, Artificial Analysis, and LiveBench all have submission paths. Whether Subquadratic submits, and how the model performs under benchmark conditions it didn’t run itself, is informative on its own.

Third, the gated rollout. One observation from the launch criticism deserves more weight than it got: if SubQ truly reduces compute by 1,000x and costs less than 5% of Opus, the company should have no trouble scaling access. A long waitlist is what you’d expect if the economics are worse than claimed, not 1,000x better. The pace at which access opens up is itself a signal.

Fourth, the Magic.dev test. The two companies will look more or less similar over the next twelve months. If SubQ ships at the claimed price, lands open leaderboard results, and gets used outside the company, the parallel breaks. If it stays gated, stays vendor-benchmarked, and the 12M numbers never get reproduced, the parallel holds. Either outcome is information.

For now, the honest read is the one Whedon’s own admission supports: SubQ is a real sparse-attention contribution layered on top of an existing open-source base, with strong vendor-run numbers, one third-party kernel review, and no independent reproduction of the load-bearing claims. That is interesting. It is not yet what the launch said it was.

I’ll update this article if the technical report lands, the weights ship, or independent benchmarks change the picture.

Sources used in this breakdown include the Subquadratic launch post and technical post on subq.ai, the VentureBeat investigation by [name], coverage at DataCamp, The New Stack, Fello AI, SiliconANGLE, eWeek, aiHola, and the independent verification work tracked at Jake Cuth’s analysis. Every quantitative claim attributed to Subquadratic is from their own materials unless otherwise noted; every counterclaim is from named third parties.