Nemotron 3.5 Lightning Puts 1M-Token Agents on RTX PCs

A 30-Billion-Parameter Model That Behaves Like a 3-Billion One

NVIDIA released Nemotron 3.5 Lightning on August 11, 2026, and the interesting number is not the parameter count. It is the ratio. The model carries roughly 31.6 billion total parameters but activates only about 3.6 billion of them on any given step, which is the whole point of a hybrid mixture-of-experts design. You pay the memory cost of a mid-sized model and the compute cost of a small one.

• 30B total parameters with 3B active per token, distilled from the larger Nemotron 3 Ultra
• 1 million token context window, aimed at agents that run for hours rather than seconds
• Up to 4x higher output speed and about 30% faster agentic task completion versus comparable open models
• Released under the permissive OpenMDW-1.1 license, with builds for NVIDIA RTX PCs, workstations, data center systems, and Apple Silicon

That framing matters because the constraint on local AI agents has quietly shifted. A year ago the question was whether a self-hosted model could reason well enough to be trusted with a multi-step task. Increasingly the question is whether it can finish the task before the user gives up waiting. Lightning is engineered squarely at the second problem.

What Makes Nemotron 3.5 Lightning Fast?

Two things, working together. The first is the sparse mixture-of-experts routing that keeps active parameters low. The second is speculative decoding using multi-token prediction, a technique where the model drafts several tokens ahead and then verifies them in a single pass instead of generating strictly one at a time. When the drafts are usually correct, throughput climbs sharply without changing the output.

Independent measurements put the model near 670 tokens per second, which is fast enough that a long agent trajectory stops feeling like a batch job. NVIDIA also reports an Elo rating of 824 on agentic benchmarks, placing Lightning ahead of both gpt-oss-120b and the larger Nemotron 3 Super despite being a fraction of their size. The company is explicit that this is a speed-first model rather than a maximum-intelligence one, and that honesty is useful — it tells you what job to hire it for.

Why Does a 1M-Token Context Matter for Long-Running Agents?

Because agent failure is usually a memory problem, not a reasoning problem. A coding sub-agent that has read forty files, run a test suite twice, and parsed two stack traces has accumulated an enormous working state. Truncate that state and the agent starts repeating work it already did or contradicting a decision it made ten steps earlier.

A million-token window does not eliminate the need for good context management, but it raises the ceiling considerably. NVIDIA points at long-running personal assistants, coding sub-agents, and security operations workflows as the target uses — all cases where the session outlives any single question. It pairs naturally with the local agent hardware trend we have been tracking in our AI coverage, where the interesting work is increasingly happening on machines people already own.

Where Can You Run It Today?

Lightning is available through Ollama with a standard pull, and it plugs into agent harnesses including Claude Code, OpenClaw, Hermes Agent, and OpenCode. An MLX-optimized variant targets Apple Silicon specifically, which is a meaningful nod to the number of developers running local models on MacBooks rather than desktops with discrete cards.

The release lands one day after Meta Superintelligence Labs shipped Muse Glimmer, another 30B open model tuned for consumer GPUs, and it slots alongside NVIDIA's own NOOA agent framework as the model layer under a harness the company already open-sourced. Two 30B-class open agentic models in two days is not a coincidence — it is the size class the industry has settled on for work that has to run somewhere other than a hyperscaler.

The Practical Read

If you have been waiting for the moment when a self-hosted agent stops being a demo and starts being a tool, throughput was the missing piece. A model that answers correctly but slowly gets used once. A model that answers well enough at 670 tokens per second gets wired into a workflow. Lightning is a bet that the second one wins more often, and on current evidence it is a reasonable bet.

Sources: Ollama Blog — August 11, 2026; NVIDIA Blog — August 11, 2026; NVIDIA Technical Blog — August 11, 2026; The Decoder — August 11, 2026.