How Much RAM Do You Need for Local AI Agents?

Say you've got an AI model running on your PC and chatting with it works fine. Now you want to hand it a coding job, where it reads through your project files and makes changes on its own. You'd expect that to work just as well, but it often needs a lot more memory, because the model has to keep far more text in mind at once. That's called the context window, and every bit of text in it uses memory on top of the model itself.

It adds up fast. With Qwen3-Coder-30B-A3B, the context size Ollama recommends for this kind of job (64K tokens, roughly the text of a short novel) makes the model need about a third more memory. At the largest context it can handle, 256K tokens, the text alone needs more memory than the model does.

So when you're working out how much RAM you need to run AI agents locally, count the model and the context together, and make sure all of it fits in your graphics card's memory. Some machines, like Macs & DGX Spark, share one pool of memory between the processor and the graphics, and then that shared pool is what needs to be big enough. After that, leave some regular RAM free for the coding app itself.

Every model works inside a context window, which is how much text it can see at once, counted in tokens (a token is a word or part of one). An agent works in a loop: it reads your request, runs a command or a search, reads what came back, opens a file and goes round again, and every step lands in that window. As Anthropic puts it, "An agent running in a loop generates more and more data that could be relevant for the next turn of inference."

That's why the apps that run models locally ask for big windows once you connect an agent. Ollama's context length page says agents and coding tools "should be set to at least 64000 tokens," and its guides for Claude Code and OpenClaw say the same. LM Studio wants more than about 25k for Claude Code and Codex, about 50k for OpenClaw and about 64k for Hermes Agent.

It's important to note that the default is often far smaller. Default depends on your GPU memory, and on cards under 24 GiB it's just 4k tokens. So an agent only takes up more memory once you raise that setting, and on a smaller card it'll keep running out of room until you do.

Your graphics card's memory has to hold the model itself, which takes about as much room as the download, and the context. The app running the model (Ollama, LM Studio or llama.cpp) keeps the context in something called the KV cache, a running record of every token in the window so the model doesn't have to work through them all again on each step. The app also needs a bit of working space. System RAM has a different job, holding the OS, the agent app and its tools.

On a PC with a graphics card, the card's own memory (VRAM) and system RAM are separate pools, so you can't add them together and call it one budget. On a unified memory system, where the processor and graphics share memory, everything draws from one pool. Our guides to memory for local LLMs and GPU memory for local AI cover sizing the model itself, so here we'll stick to what the agent adds.

You can work this out yourself with NVIDIA's formula for the KV cache. Open the model's config file and find three numbers: the number of layers, the number of "KV heads" and the head size. Multiply them together, then multiply by four (two because the cache stores two things per token, and two bytes per number at the default 16-bit setting). That gives you the memory each token takes, and you multiply by the length of your context window.

Let's put a real coding model through it. Qwen3-Coder-30B-A3B-Instruct is a mixture-of-experts model (only some of its parameters work on each token) with 30.5B parameters in total and 3.3B active. Its config lists 48 layers, 4 KV heads and a head size of 128, with a native window of 262,144 tokens. By our math, that's 96 KiB of cache for every token at f16 (16-bit, the default setting in Ollama and llama.cpp). Ollama lists the standard 4-bit Q4_K_M download at 19GB, which is about 18.6 GB to the byte.

These are our numbers from Qwen's published config, for one request at a time, before the app's working space. Qwen's model card even suggests "reducing the context length to a shorter value, such as 32,768" if you run out of memory.

Don't assume a bigger model always means a bigger cache, though. By the same math, the regular (non-mixture-of-experts) Qwen3-8B uses 144 KiB per token, more than the 30B coding model, so at the 40,960-token default in Qwen's config the cache comes to about 6.0 GB, more than its 5.2GB download.

Context is the main cost, but it isn't the only one. In Ollama, OLLAMA_NUM_PARALLEL sets how many requests each model handles at once (the default is 1), and the FAQ says "a 2K context with 4 parallel requests will result in an 8K context and additional memory allocation." Set it to 4 with our example at 64K and, by our math, that's about 25.8 GB of f16 cache on top of the 19GB model.

LM Studio has a similar setting, Max Concurrent Predictions, which defaults to 4. Extra models count in full, too: LM Studio's OpenClaw guide has it load a second model to power OpenClaw's memory search, and that sits in memory next to the one doing the talking.

If the numbers don't fit, you've got a few things to try before you buy anything. The biggest is shrinking the KV cache by storing each number in fewer bits, which Ollama calls quantizing it. Ollama's FAQ says q8_0 "uses approximately 1/2 the memory of f16 with a very small loss in precision" and q4_0 about a quarter "with a small-medium loss in precision that may be more noticeable at higher context sizes." You set it with OLLAMA_KV_CACHE_TYPE, and Ollama's Flash Attention setting has to be switched on too.

Going by those rough fractions, our example at 64K drops from about 6.4 GB of cache to roughly 3.2 GB at q8_0, or 1.6 GB at q4_0. We'd try q8_0 first.

Sam McLeod, who wrote the code that added the feature to Ollama, measured it in December 2024 on Qwen 2.5 Coder 7B and found q8_0 made very little difference against f16. He hadn't tested q4_0 on that model, but he says his earlier testing showed a noticeable drop in quality with it. In llama.cpp, the same setting is split into -ctk and -ctv for keys and values.

Beyond that, set the window your agent asks for instead of the model's full one (going from 256K to 64K saves about 19 GB on our example), keep parallel requests at 1 unless you really need them, and unload models you aren't using.

There isn't one number for "agents," because it depends so much on the model. What we'd do is work it out in three steps:

1. Take the model file size for the build you'll run.
2. Add the KV cache at the context length your agent asks for (64K is a sensible start, as that's what Ollama recommends), times the number of parallel requests.
3. Leave headroom for the app's working space, and keep the total inside your graphics card's memory, or the part of shared memory the graphics can use.

Then check it for real before you trust it. In Ollama, ollama ps shows a CONTEXT column with the context it allocated and a PROCESSOR column, which Ollama's docs say you want reading "100% GPU" instead of splitting the model with the CPU. In LM Studio, lms load --estimate-only prints a memory estimate without loading the model.

System RAM is harder to pin down. OpenClaw's install page, for example, lists Node and a supported OS but no memory figure. So leave room for your OS, the agent and whatever it opens, like a browser, and don't count on spare system RAM to rescue a model that doesn't fit in your graphics card's memory.

Sticking with our Qwen3-Coder example, here's how it works out by our math. Graphics card memory is counted in binary gigabytes (GiB), which are about 7% bigger than the decimal ones in our table, so in card terms the model plus a 64K f16 cache is about 23.3 GiB.

A single 32GB card has room for that. The FlexPrime V20R comes with an NVIDIA RTX PRO 4500 Blackwell with 32GB of GDDR7, plus 32GB of DDR5 as separate system RAM, which leaves about 8.7 GiB of the card for the app's working space or a longer window.

On a 24GB card, the same setup takes nearly all of the memory before the app's working space, so you'd want to shorten the context or quantize the cache.

For the full 256K window, several requests at once or a second model, a big shared pool gives you the room. The CORSAIR AI Workstation 300 has 128GB of LPDDR5X, and up to 96GB of it can be allocated to graphics. That's shared memory, not dedicated VRAM, so the OS and your agent app come out of the same 128GB. The NVIDIA DGX Spark, an NVIDIA system we sell, has 128GB of memory shared between its processor and graphics. On either one, the full window plus our example model comes to about 44 GB.

With DGX Spark in particular, that's a step up in capacity, not an upgrade across the board: its memory bandwidth (how fast data moves in and out of memory) is listed at 273 GB/s, where the RTX PRO 4500's is 896 GB/s. Our piece on shared memory and mixture-of-experts models goes further into how a shared pool works.

Before you spend anything, though, load the model you want at a 64K context on the hardware you've already got and see what ollama ps says. If it doesn't read 100% GPU at the context your agent needs, try the tricks above first, and if it still won't fit, that's when it's time to look at our AI workstations.