To run a chat or coding model on your own PC, you need GPU memory for the model, the conversation, and a little working space for the app. The model alone takes about as much memory as the file takes on disk. The conversation adds to that: the app keeps your prompt and recent replies in memory, and a longer chat takes more.
A 12GB card usually means a smaller model and a shorter chat. A 24GB or 32GB card can run a bigger model, a longer chat, or both. If the total is more than the card has, the model will not load, or it slows down.
The model file is what model pages call the weights. For GGUF downloads, the listed file size is a solid guide to how much GPU memory those weights need.
The space for your prompt and chat history is the KV cache (often just called context). It grows with the history you keep and with the context limit you set. A large reserved limit still uses that space even on a short question.
The rest is working space for the app: buffers, drivers, temporary workspace. You see that in the load log or a monitor such as nvidia-smi, not on the product page.
You can often shrink the same model into a smaller file. That is quantization: a Q4 file is smaller than a Q8 or full-precision copy. Quality can change, and two files with the same bit label can still differ, so use the exact filename and size.
The conversation side can use as much memory as the model file. Longer reserved history means a larger KV cache. Some models need less for long chats because only part of their layers grow that cache. Qwen3.5-27B, for example, has 64 layers but only 16 that grow KV cache as history gets longer.
On a system with one shared memory pool for CPU and GPU, ordinary tools show that pool while a model is loaded:
From about 12GB to 48GB, many local coding setups stay on one mid-size model family (around 27 billion parameters) and change how compressed the file is, or how long the chat can be, instead of jumping to a different model.
| GPU memory | Typical approach | What that usually means |
|---|---|---|
| 4-6 GB | Small models | Fine for quick, single-file questions. Multi-file agent work usually does not fit. |
| 8 GB | Split across GPU and system RAM | A full 27B-class model on the GPU alone is often too slow. Keep some of it in system RAM, and leave enough RAM for that. |
| 12-16 GB | Same model family, smaller file | A common 4-bit file can exceed 16GB before chat history. Smaller 3-bit or 2-bit files are common; use the exact upload size. |
| 24 GB | 27B at about 4-bit | The usual fit for that class with useful context. A very large reserved window can still fill the card. |
| 32 GB | Same 4-bit, more context | Room for longer documents or chats. Matches dedicated GPU memory on systems such as the FlexPrime V50R. |
| 48 GB | Same family, higher precision | Often an 8-bit build of the same model, with more headroom. |
| 128 GB unified | Larger MoE models | Practical on platforms such as DGX Spark. Fit still depends on reading speed and what else shares the pool. |
Across the middle of that range, a bigger card usually means a longer usable chat for the same model, not a completely different model.
Take Meta-Llama-3.1-8B-Instruct Q4_K_M GGUF. Hugging Face lists the file at 4.92 GB. That is roughly the memory for the model file alone.
Conversation space grows with how much history you reserve. For this model, that works out to about 128 KiB per reserved token (from its layer and attention layout, with an f16 KV cache).
| Reserved context | Conversation space | Weights + conversation |
|---|---|---|
| 4,096 tokens | ~0.5 GB | ~5.4 GB |
| 8,192 tokens | ~1.0 GB | ~5.9 GB |
| 32,768 tokens | ~4.0 GB | ~8.9 GB |
The app’s working space sits on top of these figures. Your load log shows the full total.
On a discrete-GPU workstation, GPU memory and system RAM are separate.
On the FlexPrime V50R, the NVIDIA RTX PRO 4500 Blackwell has 32GB of dedicated GPU memory. The system also has 64GB of DDR5. Models on the GPU use the 32GB. Ubuntu and other apps use the DDR5.
On NVIDIA DGX Spark, CPU and GPU share one 128GB memory pool (coherent unified LPDDR5x). The model, the conversation, and the rest of the system all draw from it.
CORSAIR PRO builds configure-to-order AI workstations, from compact deskside systems to multi-GPU towers. Match the machine to how your software uses memory: a separate GPU pool on a discrete-GPU workstation, or one shared pool on a platform such as DGX Spark.
Explore options in the CORSAIR PRO configurator, or talk through requirements with a CORSAIR PRO solutions architect.
PRODUCTS IN ARTICLE
Stay up to date with CORSAIR. Get our latest News, Guides, and Product Updates in your Google feeds.
Add CORSAIR as a preferred sourceThanks for helping us improve this article.
JOIN OUR OFFICIAL CORSAIR COMMUNITIES
Join our official CORSAIR Communities! Whether you're new or old to PC Building, have questions about our products, or want to chat about the latest PC, tech, and gaming trends, our community is the place for you.