How Much RAM Do You Need to Run a Local LLM in 2026?

The Short Answer

If you take one rule away from this guide, take this one: your combined RAM plus VRAM needs to be a little larger than the model file you downloaded. Not the parameter count. Not the original weights. The size of the specific quantised file on disk, plus headroom.

Everything below is that rule, applied.

Quick Picks

• 8GB total memory — an 8B model at 4-bit, or a 27B at aggressive 1-bit quantisation. Fine for chat and summarisation, tight for anything else
• 16GB — a 14B at 4-bit comfortably, or a 27B at 2-bit. The realistic floor for a daily driver
• 24GB — a 27B-class model at 4-bit with room for context. The sweet spot for a single consumer GPU
• 32 to 64GB — a 27B at 6-bit, or a 70B-class model at 4-bit with careful context budgeting
• 128GB — 100B-class mixture-of-experts models at 3-bit to 4-bit, which is where most serious self-hosted work now happens
• 192GB — comfortable 4-bit headroom on 120B-class models plus a long context window
• 240GB and up — the entry point for trillion-parameter open models, and only at the smallest quantisations

The One Formula That Actually Matters

Model memory is not mysterious. A model's weights occupy roughly one byte per parameter at 8-bit, half a byte at 4-bit, and so on. A 27-billion-parameter model at 4-bit therefore wants somewhere in the region of 14 to 17GB, and that is almost exactly what you see in practice.

Real numbers help more than the arithmetic. Unsloth publishes dynamic GGUF quantisations for Qwen3.8-27B across the full bit-width range, and the file sizes are instructive:

• 1-bit (UD-IQ1_S): 6.19GB
• 2-bit (UD-Q2_K_XL): 9.83GB
• 3-bit (UD-Q3_K_XL): 13.1GB
• 4-bit (UD-Q4_K_M): 16.5GB
• 5-bit (UD-Q5_K_M): 19.8GB
• 6-bit (UD-Q6_K): 22GB
• 6-bit extra large (UD-Q6_K_XL): 25.3GB

One model, a 4x spread in memory footprint, depending entirely on how much precision you are willing to trade. That trade is the single biggest lever you have, and it is far more useful than shopping for a bigger machine.

What Does Quantisation Actually Cost You?

Quantisation reduces the precision of each stored weight. Fewer bits per weight, smaller file, less memory, faster loading — and some loss of accuracy.

The loss is not linear, which is the important part. Going from 16-bit down to 8-bit is close to free. Eight to four is a small, usually acceptable degradation on most tasks. Below four bits it starts to bite, and how hard depends heavily on the quantisation method. Unsloth claims its Dynamic 3.0 method delivers over 10% better top-1% accuracy than equivalently sized alternatives by selectively keeping important layers at higher precision — that is the vendor's figure for its own method, but the underlying technique of mixed-precision quantisation is well established and does measurably help at the low end.

Practical guidance: 4-bit is the default answer for almost everyone. Go to 5 or 6-bit if you have memory to spare and you care about code correctness or long-form reasoning. Go below 4-bit only when the alternative is not running the model at all — which, for the biggest models, it frequently is.

How Much Memory Does Context Use?

Weights are the fixed cost. The KV cache is the variable one, and it is what catches people out.

Every token in your context window has to be kept in memory while the model generates. At a modest 8K context this is a small addition on top of the weights. At 128K it can add several gigabytes, and on some architectures considerably more. If your model loads fine and then falls over halfway through a long document, this is almost always why.

The practical rule: budget meaningful headroom above the model file size, and if you are working with long documents, budget more. A 16.5GB model file on a 16GB card will not work. On 24GB it will work well with room for real context.

Is Unified Memory as Good as VRAM?

This is the question that decides what hardware to buy, and the honest answer is: not as fast, but often more useful.

A discrete GPU's VRAM has enormous bandwidth, and bandwidth is what determines token generation speed. A unified-memory system — Apple Silicon, or an AMD Strix Halo or Gorgon Halo box — has slower memory but far more of it, and it can hand most of that pool to the accelerator. Minisforum's new MS-S1 MAX-P495 can allocate up to 160GB of its 192GB as graphics memory. No consumer graphics card comes close.

Which matters more depends entirely on whether the model fits. A model that fits in VRAM runs fast. A model that does not fit spills to system memory or disk, and the speed penalty is brutal — Unsloth's own guidance for large models notes that falling below the recommended memory threshold can drop throughput to under 2 tokens per second because of disk offloading, against 10 or more when the model fits.

So the ranking is: fits in fast VRAM, best. Fits in slower unified memory, good. Does not fit anywhere, unusable regardless of how fast the memory is. Capacity wins ties.

How Much RAM Do I Need for a 70B Model?

Around 38 to 42GB at 4-bit, plus context headroom, so 48GB is a realistic working figure and 64GB is comfortable.

That puts a 70B model out of reach of a single 24GB consumer card at 4-bit, which is why this size class drove so much of the interest in unified-memory machines. Options that do work: a 48GB workstation card, two 24GB cards, a 64GB-or-larger unified memory system, or dropping to 3-bit and accepting the accuracy cost.

Can I Run a Frontier Open Model at Home?

Increasingly, yes — with a specific and expensive definition of "home."

The trillion-parameter open models represent the current ceiling. Unsloth's guidance for Kimi K2.5 recommends at least 240GB of unified memory or combined RAM plus VRAM for the smallest quantisation to reach 10 or more tokens per second, with its 1.8-bit build measuring around 240GB and the 2-bit version at roughly 375GB. There is also a genuinely clever path down: with all mixture-of-experts layers offloaded to system RAM, the smallest quant will run on a single 24GB GPU, and a machine with about 256GB of system RAM lands near 10 tokens per second.

That is slow by API standards and remarkable by any other measure. A model of that class running on hardware you own, with no data leaving the building, was not a realistic proposition eighteen months ago.

For most people the better target is a level down. A 100B-class mixture-of-experts model at 3-bit to 4-bit fits in 128GB and covers the overwhelming majority of self-hosted use cases — the same envelope we found when GLM-5.3-Flash ran on 128GB of local RAM last month.

Choosing Your Tier

Work backwards from the model, not forwards from the budget.

Pick the model you actually want to run. Find its 4-bit file size. Add 25 to 40% for context and overhead. That is your memory target. Then decide whether you are buying capacity — a unified-memory mini PC or workstation — or bandwidth, meaning a discrete GPU with enough VRAM to hold the whole thing.

If you want a shortcut: 24GB of fast VRAM covers most single-model workflows. 128GB of unified memory covers most multi-model and long-context workflows. Above that you are in specialist territory, and you should be sizing against a specific model rather than a general aspiration.

Our buyer's guide to mini PCs for local LLMs covers the machines themselves, and there is more hardware analysis across our mini computer coverage.

Sources: Unsloth — Kimi K2.5 run locally guide — 2026; Hugging Face — unsloth/Qwen3.8-27B-GGUF model card — August 2026.