Nvidia squeezes a 74B open model into 24GB of memory
Nemotron 3.5 Bolt 74B-A9B is a mixture-of-experts model built to run locally, with day-one support in llama.cpp, Ollama, vLLM, Unsloth and LM Studio.
Nvidia released Nemotron 3.5 Bolt 74B-A9B today, an open-weight model built to run on a single machine rather than behind an API. It’s a mixture-of-experts design: 74 billion parameters in total, but only 9 billion active per token, and Nvidia compresses the whole thing down to a 24GB memory footprint using 2-bit quantisation paired with quantisation-aware training to limit the quality loss.
It ships with support from launch in llama.cpp, vLLM, Ollama, Unsloth and LM Studio, as well as Nvidia’s own Hermes and OpenClaw agent apps. Training data, weights and recipes are all published, so it can be fine-tuned rather than just run as-is. Nvidia pitches it for always-on personal agent work: coding, research, monitoring and tool use. The release is timed just ahead of Nvidia’s RTX Spark laptops, shipping tomorrow with up to 128GB of unified memory, and a second, much smaller 4GB model aimed at in-game NPCs follows on 23 October.
Why it matters: a 74B-class model that fits in 24GB means it runs on a single high-end consumer GPU instead of needing multiple cards or a cloud API call, and the open weights mean it can be fine-tuned into a specific agent rather than used only as Nvidia shipped it.
The caveat: Nvidia’s claim that it’s “faster than most popular dense models” and “smarter than most popular MoE models” of similar size comes with no benchmark numbers attached, so it isn’t independently verifiable yet.