NVIDIA Just Shipped a 30B AI Model With Only 3B Active Parameters. It’s Built for AI Agents

I almost skipped this release.

gemini ( nano banana )

Another week, another “open weight” model dropping into my feed with a chart claiming it beats everything twice its size. I have written enough of these articles to be numb to the pattern. Then I actually opened NVIDIA’s own benchmark table instead of the announcement thread, and the honesty of it stopped me.

Here,s Friend link https://medium.com/@inprogrammer/nvidia-just-shipped-a-30b-ai-model-with-only-3b-active-parameters-its-built-for-ai-agents-ba9a39aec6b5?sk=0f840ef8b29008d60f7021a54bcba6af

NVIDIA is not claiming Nemotron 3.5 Lightning is smart. They are claiming it is fast, cheap, and built for a job most coverage of AI models still ignores: being the worker, not the thinker, inside an agent pipeline.

That distinction is the whole article.

If you’re building agent automations for clients and watching your OpenRouter bill more closely than your benchmark charts, this one’s for you.

The 30-second version: Nemotron 3.5 Lightning is a 30B open model that only activates 3B parameters per token, so it’s cheap and fast but weak at complex reasoning. Use it for high-volume, well-scoped agent steps, tool calls, formatting, validation. Don’t use it for coding agents or anything needing sustained multi-step reasoning; GLM 5.2, Kimi K3, or Muse Glimmer will outperform it there. On OpenRouter it runs roughly 15 to 50 times cheaper per task than those three for the simple work it’s built for. If you’re already routing sub-agent traffic through one of them, keep doing that for the hard steps and consider Lightning only for the mechanical ones underneath.

What Is NVIDIA Nemotron 3.5 Lightning

Nemotron 3.5 Lightning is an open weight language model NVIDIA released on August 11, 2026. It has 30 billion total parameters but only activates about 3 billion of them per token, thanks to a Mixture of Experts design layered on a hybrid Mamba 2 and attention backbone. NVIDIA built it specifically for always on AI agents that run high volume, repetitive tasks like tool calls, validation, and formatting rather than deep reasoning.

It is the successor to Nemotron 3 Nano 30B and, according to its listing on OpenRouter, a distillation of the much larger 550 billion parameter Nemotron 3 Ultra that NVIDIA shipped in June. It went live the same day on Hugging Face, Together AI, DeepInfra, Fireworks, and OpenRouter.

Here is the spec sheet in plain terms:

Why the 30B/3B Split Actually Matters

The “A3B” in the model’s name is the entire pitch. You pay the memory cost of a 30 billion parameter model but the compute cost of a 3 billion parameter one, because a router only activates a small slice of experts for each token.

Stack that on a Mamba 2 backbone and it gets more interesting. A normal transformer’s memory use grows with every token in the context window, which is why long context serving usually falls apart under concurrency before it falls apart under context length. Mamba layers carry a fixed size state instead, so most of that 1 million token window costs constant memory rather than scaling memory. Only the attention layers NVIDIA interleaves into the stack pay the usual price.

This is the same architectural direction the whole open weight field has been converging on for about a year. Sparse activation plus linear state sequence mixing is quietly becoming the default recipe for models meant to run cheap and fast rather than win leaderboards outright.

The Benchmarks, Read Honestly

NVIDIA published numbers for both the full precision BF16 model and the quantized NVFP4 checkpoint it actually released to the public. I am not going to just paste the highlight reel. Here is the full table, and then what it actually tells you.

Three things jumped out once I sat with these numbers instead of the launch tweet.

First, MMLU Pro at 81.94 and GPQA Diamond at 75.44 are genuinely strong for a model activating only 3 billion parameters per token. On knowledge and single shot reasoning, this thing punches well above its activation budget. Artificial Analysis places it near gpt oss 120b on their Intelligence Index, at roughly a quarter of the total parameters.

Second, the agentic long horizon numbers are weak, and NVIDIA does not hide it. Terminal Bench 2.1 at 24.58 is not competitive. For comparison, Meta’s Muse Glimmer 30B, released a day earlier at a similar size, scores 51.7 on the same benchmark. A model that hits 85 on PinchBench and 24 on Terminal Bench is telling you something specific. It executes short, well scoped steps reliably and struggles the moment a task requires sustained autonomous reasoning over many steps.

The chart above puts that gap next to the two open-weight models most of my readers already route through, plus Glimmer. It’s a wide gap, not a rounding error, and it’s the single number that should drive the “should I switch” decision later in this piece.

Third, some NVFP4 scores actually rose after quantization, like SWE bench Verified going from 51.56 to 52.80. That is not the model getting smarter from compression. It is run to run noise, and it is a useful reminder not to read too much into a single point of movement anywhere on these tables.

Speed Claims, Checked Against the Source

NVIDIA says Nemotron 3.5 Lightning delivers up to 4x the output speed of similarly sized models, measured in tokens per second. OpenRouter separately reports up to 30 percent faster wall clock completion on full agent task trajectories. Those are two different measurements and both carry an “up to” qualifier worth taking seriously.

NVIDIA’s own launch chart plots roughly 17 H100 GPU hours to finish 10,000 PinchBench tasks against about 24 hours for Qwen3.6 35B and 25.5 hours for Gemma 4 26B. That works out to roughly 29 to 33 percent faster, which lines up with OpenRouter’s figure. It also concedes something the marketing copy does not say out loud: Qwen3.6 35B actually sits slightly higher on raw PinchBench accuracy.

Lightning’s win is on the time axis, not the intelligence axis.

In independent pre-release testing, Artificial Analysis clocked Lightning at close to 669 tokens per second, the fastest of any model in its comparison set and nearly double Gemini 3.5 Flash-Lite’s 386 tokens per second.

Throughput isn’t the whole latency story, though. For anything interactive, a chat-facing agent rather than a batch job, time to first token matters as much or more than raw generation speed. Here the picture is thinner than I’d like. Uniphore benchmarked Lightning against a same-size peer, Gemma 4 31B, in their own production evaluation stack and measured a 4.9x time-to-first-token advantage, holding consistently across three different input and output token profiles. That’s a real result, but it’s a relative number against one same-size peer, not an absolute figure I can put next to GLM 5.2 or Kimi K3’s own published TTFT numbers. I don’t have an apples-to-apples TTFT comparison across all four models in this piece, and I’d rather say that plainly than paper over it with a number that isn’t directly comparable.

How This Fits in a Planner, Worker, Critic Pipeline

Before the comparison numbers, it helps to see the shape of the thing NVIDIA is actually proposing. Lightning isn’t meant to run alone. It’s meant to sit inside a pipeline where a stronger model plans, Lightning executes, and a critic checks the work before anything ships.

(Diagram: planner model plans the task, hands off through NeMo Switchyard, which routes execution to Lightning and quality checks to a critic model; failed or low-confidence steps loop back through Switchyard for a retry. If you’re pulling this into the Medium draft, either screenshot the interactive version generated for this piece, or hand a designer this description: four labeled boxes, purple “Planner” at top, gray “NeMo Switchyard” beneath it, teal “Worker: Lightning” and coral “Critic” side by side at the bottom, arrows flowing top to bottom and a dashed loop-back line from Critic to Switchyard.)

Keep that shape in mind for the next two sections. It’s the frame the rest of this article hangs on.

How This Compares to What You Might Already Be Running

GPU hours are an abstraction if you’re not self-hosting. Most of my readers are routing models through OpenRouter, so the question that actually matters is simpler: if you’re already sending your sub-agent traffic through Muse Glimmer, GLM 5.2, or Kimi K3, is Lightning worth switching to?

Short answer: not for the same job. Here’s the honest comparison, specs and cost in one place:

chatgpt

That Terminal-Bench gap is not a rounding error. GLM 5.2 and Kimi K3 are both landing in the high 70s to low 80s on the same benchmark where Lightning scores 23.46. If your sub-agents are doing anything that resembles a real coding or terminal task, GLM 5.2 or Kimi K3 will finish the job and Lightning frequently will not. Don’t switch off either of them for that layer of work.

Muse Glimmer is the interesting middle case. Same parameter count as Lightning, dense rather than sparse, and it more than doubles Lightning’s Terminal-Bench score at roughly three times the price. If your sub-agent work sits between “trivial tool call” and “full coding task,” Glimmer is worth evaluating before you reach for GLM 5.2 or Kimi K3, since it’s still self-hostable on a single consumer GPU and dramatically cheaper than either of the two larger models.

Where Lightning actually competes is upstream of that comparison entirely: the tool-call formatting, validation, and short structured-output steps that don’t need 80 points on Terminal-Bench, they need to run thousands of times an hour without falling over on cost. For a batch of tasks running roughly 500 input tokens and 300 output tokens each, a rough profile for a scoped validation or tool-call step, that’s roughly 16x cheaper than GLM 5.2 and close to 48x cheaper than Kimi K3.

Scale it up: route 10,000 support ticket triage calls a day, each one just reading a ticket and returning a category, priority tag, and routing decision, through Lightning instead of GLM 5.2, and you’re looking at roughly $1.25 a day instead of about $20. At today’s exchange rate that’s close to ₹120 a day versus roughly ₹1,900. Over a month, for an Indian freelancer or small agency billing a client for an automation build, that’s the difference between spending about ₹3,600 and about ₹57,000 on infrastructure for work that was never complex enough to need a 750-billion-parameter model reasoning about it in the first place. That gap alone can be the difference between a project being profitable at a typical retainer and eating the entire retainer in API costs.

The actual decision isn’t Lightning versus everything else. It’s whether you’re paying frontier-adjacent prices for work that never needed them.

If yes, that’s where Lightning earns its place, not as a replacement for your coding sub-agent, but as the layer underneath it.

What About Prompt Injection and Tool-Access Safety

This is the section most coverage of the launch skipped entirely, and it matters more than the benchmark tables once you’re actually wiring a model up to email, banking, or CRM tools rather than just chatting with it.

I went looking for a Lightning-specific number on the standard agent security benchmark, AgentDojo, which measures how often an agent can be hijacked into calling a malicious tool when an attacker’s instructions are hidden inside data the agent reads, an email, a search result, a calendar invite. NVIDIA has not published an AgentDojo or equivalent prompt-injection result for Lightning specifically, at least not anywhere I could find as of this writing.

Meta, by contrast, did publish this for Muse Glimmer: a 28.4% attack-success rate on an adaptive AgentDojo variant, versus 25.6% for Gemma and 40.3% for Qwen in the same test, while posting the highest task-utility score of the three. That’s a real, if imperfect, number you can weigh against a competitor’s model before you deploy it.

The honest takeaway is that this is a gap in Lightning’s public documentation, not necessarily a sign it’s less safe than the alternatives. It’s worth flagging as a genuine due-diligence item: if you’re planning to give Lightning tool access to anything with real consequences, ticket systems, financial data, customer records, run your own AgentDojo-style red-team pass, or wait for NVIDIA to publish one, rather than assuming the absence of a bad number means a good one. The lower a model sits in your agent stack, the more raw, untrusted data it tends to touch directly, tool outputs, scraped content, user-submitted text, which is exactly where injection attacks land.

How to Actually Deploy It

The checkpoint NVIDIA released is nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4. It is already post training quantized, so there is no separate full precision release to download. It runs on a single H100 or on an NVIDIA DGX Spark (GB10).

For interactive workloads, NVIDIA recommends switching on its DSpark speculative decoding method and keeping concurrency at 128 requests or fewer:

bash

export DSPARK_CKPT=nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-DSpark

vllm serve --model $MODEL_CKPT \
    --max-num-seqs 128 \
    --enable-prefix-caching \
    --async-scheduling \
    --speculative_config.model $DSPARK_CKPT \
    --speculative_config.num_speculative_tokens 3 \
    --mamba-ssu-algorithm horizontal \
    --mamba-backend flashinfer \
    --mamba-ssm-cache-dtype float16 \
    --reasoning-parser nemotron_v3 \
    --tool-call-parser qwen3_coder \
    --enable-auto-tool-choice

One gotcha that will bite you if you skip the model card. For coding agents, you need to pass force_nonempty_content in your request, or you will get tool calls back with empty message content that most agent harnesses reject outright.

python

response = client.chat.completions.create(
    model=MODEL,
    messages=[{"role": "user", "content": "What's the weather in Santa Clara?"}],
    tools=tools,
    max_tokens=16000,
    temperature=1.0,
    top_p=0.95,
    extra_body={"chat_template_kwargs": {"force_nonempty_content": True}},
)

Teaching It Your Own Workflow: A Real Example

NVIDIA’s model card claims Lightning is easy to post-train with LoRA or full SFT through NeMo AutoModel. CodeRabbit actually did this, and published enough detail that it’s worth walking through rather than taking on faith.

The task was routing code reviews: reading a code change and deciding which specialized reviewer model should handle it, matching CodeRabbit’s own internal routing policy. They started with supervised fine-tuning using NeMo AutoModel, training a lightweight rank-8 LoRA adapter for a single epoch on a distilled dataset, reference routing decisions generated by a stronger model, then filtered. On a frozen 1,000-task evaluation, that alone moved exact route agreement from 75.8% to 80.4%.

Then they layered reinforcement learning on top using NeMo RL, running GRPO against CodeRabbit’s actual routing policy for 910 selected prompts. Cohen’s kappa, a stricter agreement measure than raw accuracy, moved from 0.461 to 0.544.

The finished adapter is a single rank-16 LoRA sitting on stock Lightning, serving on one A100. Measured throughput was 314.82 aggregate output tokens per second across eight concurrent requests. Against the untuned baseline, the tuned model ran at roughly half the cost per task and produced 63.4% fewer output tokens getting to the same answer.

That’s the concrete version of what “customizable” means in NVIDIA’s pitch. If you’re building an automation for a client, an appointment triage bot, a lead qualification flow, a document classification pipeline, that’s the shape of the investment: a rank-8 or rank-16 LoRA, a training set built by distilling from a stronger model’s outputs, a single GPU to train and serve on. It’s a weekend project for someone comfortable with NeMo, not a research undertaking.

Where This Model Is the Wrong Choice

Most coverage of this launch skipped this part, and it is the part that actually saves you time.

• Do not make it your primary coding agent. A 24.58 on Terminal Bench 2.1 means long autonomous terminal sessions will stall out, and GLM 5.2 or Kimi K3 will simply do better work at that layer.
• Do not route complex, policy constrained multi turn tool workflows through it. The 9.28 on tau cubed bench Banking reflects that directly.
• Do not treat the full 1 million token context as free at high concurrency. Mamba layers keep most of it constant cost, but the interleaved attention layers do not, and that is where memory pressure shows up first.
• Do not read the Artificial Analysis Intelligence Index score of 24 as frontier adjacent. It is a strong number per active parameter. It is not a strong number in absolute terms, and NVIDIA never claims otherwise.
• Do not give it tool access to anything high-stakes without your own red-teaming pass. See the safety section above; the public documentation gap is real.

Where it actually earns its place is the worker tier of the planner, worker, critic pipeline sketched out earlier in this piece. Use a stronger frontier model to plan and reason, then hand the high volume execution, retrieval, summarization, and repetitive tool calling to Lightning.

NVIDIA didn’t leave that pattern as an exercise for the reader. Alongside Lightning, it released NeMo Switchyard, an open source routing library built to do exactly this: sit in front of your agent stack and send each step of a task to whichever model, open, proprietary, or NVIDIA’s own, actually fits the job, without you rewriting your application logic every time a step’s complexity changes mid-task. Cognition has already plugged Switchyard’s router into Devin Desktop and reported holding near-frontier task completion while cutting mean cost by roughly 28 percent versus routing everything through a single frontier model. If you’re building the planner-worker-critic pattern by hand today, calling model APIs conditionally based on your own heuristics, Switchyard is worth a look before you keep maintaining that routing logic yourself.

NVIDIA is explicitly pitching post training fine tunes for coding, legal, and cybersecurity work on top of the Lightning base, which is where I would actually spend time with it if I were building an agent stack today.

How NVIDIA Trained It

Worth noting for anyone evaluating the training data provenance. NVIDIA used a five stage pipeline: pretraining on over 20 trillion tokens with an NVFP4 recipe in Megatron LM, continued pretraining for its multi token prediction heads, supervised fine tuning on code, math, science, and tool calling data, reinforcement learning using GRPO across multiple task environments, and finally the post training quantization pass.

Two details are worth flagging. NVIDIA used gpt oss 120b, itself an open weight model, as the teacher for generating synthetic reasoning traces and verified CUDA kernels. And NVIDIA disclosed that it applied keyword and regex based filters to strip out reasoning traces from teacher models that implicitly aligned with specific political entities or nationalistic narratives.

That is an unusually specific admission for a launch document, and it is worth being clear about what it does and doesn’t mean for you if you deploy this. It is not, on its own, a reason to avoid the model. Every synthetic-data pipeline inherits some amount of the teacher model’s leanings, and naming the failure mode instead of quietly ignoring it is a point in NVIDIA’s favor on transparency. What it does mean is that the filtering was keyword and regex based, which is a blunt instrument. That kind of filter can both miss subtler framing it wasn’t tuned to catch and over-remove legitimate content that happened to trip a keyword. For a model doing tool calls, formatting, and validation, that’s a low-stakes gap. If you’re fine tuning Lightning for anything touching legal, policy, journalism, or other judgment-sensitive text generation, it’s worth running your own spot checks on outputs rather than assuming the base filtering covered it.

Frequently Asked Questions

Is Nemotron 3.5 Lightning open source? It is open weight under NVIDIA’s OpenMDW 1.1 license, which explicitly permits commercial use. It is not an OSI approved open source license and it is not Apache 2.0, so check the actual terms if you are in a regulated environment.

Can I run Nemotron 3.5 Lightning on a single GPU? Yes. The released NVFP4 checkpoint runs on one H100 or on an NVIDIA DGX Spark with a GB10 chip.

Is Nemotron 3.5 Lightning good for coding agents? It works well as a sub agent handling scoped, well defined coding subtasks. It is not strong enough on Terminal Bench 2.1 to serve as your primary autonomous coding agent.

How does it compare to Qwen3.6 35B? Qwen3.6 35B scores marginally higher on raw PinchBench accuracy, but Lightning completes the same 10,000 task batch roughly 30 percent faster on H100 GPU hours.

Is Nemotron 3.5 Lightning better than GLM 5.2 or Kimi K3 for agents? Not for the same work. GLM 5.2 and Kimi K3 both score roughly 80 points on Terminal-Bench 2.1 versus Lightning’s 23.46, so they’re the better choice for anything resembling real coding or terminal agent work. Lightning’s advantage is cost and speed on simpler, high-volume steps like tool-call formatting and validation, where it can run 15 to 50 times cheaper per task than either of them.

Is Nemotron 3.5 Lightning safe to give tool access to? NVIDIA hasn’t published an AgentDojo or equivalent prompt-injection benchmark for Lightning as of this writing, which is a real documentation gap rather than a confirmed problem. If you’re deploying it with access to sensitive tools, run your own red-team pass rather than assuming it’s covered.

The Real Takeaway

Nemotron 3.5 Lightning is not trying to be your next frontier model, and NVIDIA is refreshingly upfront about that. It is a cheap, fast worker model built for the unglamorous part of agent systems, the thousands of small tool calls and validation steps that do not need a genius, they need a model that shows up fast and does not fall over at scale. If that is the layer of your stack you are optimizing, this is worth a serious look. If you were hoping for a smaller model that reasons like a frontier one, keep looking.

If you’re already running GLM 5.2, Kimi K3, or Muse Glimmer for your agent work, I’d genuinely like to know: are you routing everything through one model, or have you started splitting planner and worker roles the way this piece describes? Drop it in the comments, that’s the workflow question I think most people building agents right now haven’t fully answered for themselves.

If you found this breakdown useful, follow for more honest reads on the open weight releases actually worth your attention this year.