Nemotron 3.5 Bolt: Can a 74B Open Model Really Fit in 24GB?

Nemotron 3.5 Bolt is the most interesting open model announcement of the week for anyone who wants capable AI running on their own machine. On October 13, NVIDIA said the 74-billion-parameter model, which uses a mixture-of-experts design with 9 billion active parameters, has been packed into just 24GB of memory using 2-bit quantization-aware training. The weights arrive Thursday, October 15, with support in the most popular local AI runtimes on day one.

• Size: 74B total parameters, 9B active per token, mixture-of-experts architecture.
• Memory: fits in 24GB thanks to 2-bit quantization-aware training (QAT), according to NVIDIA.
• Release: weights, training data and recipes publish on Thursday, October 15, 2026.
• Runtimes at launch: llama.cpp, vLLM, Ollama, Unsloth and LM Studio, plus direct access from the Hermes and OpenClaw agent platforms.

What Is Nemotron 3.5 Bolt?

Nemotron is NVIDIA's family of open models, and the 3.5 generation has been aimed squarely at agents. We covered its sibling, Nemotron 3.5 Lightning, in August. Bolt takes a different tack: rather than chasing the smallest possible model, it keeps a large 74B parameter count and squeezes it into the memory budget of a single high-end consumer GPU or a well-equipped laptop.

The mixture-of-experts design is what makes this practical for speed. Only around 9B parameters activate for each token, so the model generates text at a pace closer to a mid-size dense model while drawing on the knowledge stored across all 74B parameters.

NVIDIA describes Bolt as "fully open source," meaning the weights, the training data and the training recipes are all being published. Open data and recipes let researchers reproduce and audit the work, a higher bar than open weights alone.

Why Does 2-Bit Quantization-Aware Training Matter?

Quantization shrinks a model by storing each weight in fewer bits. Most local AI users already run 4-bit or 5-bit quantized models, because full 16-bit weights for a model this size would need far more memory than any consumer card offers. TechTimes notes that a comparable dense, full-precision model would need 140GB or more.

The usual problem with going all the way down to 2 bits is quality: squeezing an already-trained model that hard tends to damage its reasoning. Quantization-aware training addresses that by teaching the model to work with low-precision weights during training, rather than compressing it afterward. NVIDIA says the result is a model that is faster than most popular dense models and smarter than most popular MoE models with a similar memory footprint. NVIDIA has not yet published benchmark numbers alongside that claim, so independent testing after Thursday's release will be the real proof.

Where Can You Run Nemotron 3.5 Bolt?

The 24GB target is no accident. It lines up with the memory on high-end consumer graphics cards and with the unified memory on a new wave of AI laptops. NVIDIA timed the announcement alongside its RTX Spark laptops, which go on sale Friday, October 16 with up to 128GB of unified memory and up to 1 petaflop of AI compute, per NVIDIA. TechTimes reports Lenovo and Acer among the launch partners; pricing has not been published. If you are pricing up hardware for a model like this, our guide on how much memory a local AI mini PC really needs walks through the trade-offs.

Day-one support across llama.cpp, vLLM, Ollama, Unsloth and LM Studio means most hobbyists will be able to pull the model with the tools they already use. The Hermes and OpenClaw integrations point to the intended use: long-running local agents that call tools and keep data off the cloud.

What Else Did NVIDIA Ship for Local AI?

Bolt arrived as part of a larger software update from NVIDIA. Highlights from the October 13 post:

• Nemotron 3.5 Bolt E4B: a 4GB small model for NVIDIA ACE game characters, with reasoning that can be switched on or off, arriving October 23 and testable in the SkyrimNET mod.
• NVIDIA Broadcast 2.3: adds an MCP server so local AI agents can adjust effects and process files in the app.
• PAIR: NVIDIA's free, open-source Personal AI Router left beta, adding llama.cpp support and direct Hugging Face downloads.

NVIDIA also said local LLM downloads are running 5x higher than all of last year, which helps explain why it is investing so heavily in the local stack.

The Bottom Line

Nemotron 3.5 Bolt is a strong signal that big, capable open models no longer require data-center hardware. If NVIDIA's quality claims hold up under independent testing, a 74B model in 24GB could become the default "serious" local model for agent builders this winter. We will be watching the first community benchmarks closely after Thursday. For more on open models and the tools that run them, follow our AI coverage.

Sources: NVIDIA Blog, local AI on RTX Spark laptops and software updates — October 13, 2026; TechTimes, NVIDIA RTX Spark launches Friday with 74B model — October 14, 2026; Techaeris, NVIDIA RTX Spark local AI agents — October 14, 2026.