Can my computer run a local LLM?

Whether a model runs on your machine is, first of all, a memory question. The model has to fit somewhere: in system RAM, in a graphics card's VRAM, or in the unified memory that Apple Silicon shares between CPU and GPU. Quantization can shrink a model into a smaller memory budget, usually with some tradeoff in output quality. Context length and the KV cache the model keeps while it works also take memory, and both grow as a conversation gets longer.

Model file size is not the whole memory budget. Context, KV cache, runtime overhead, and background tools need headroom too.

• RAM. System memory. On a CPU-only machine, the model and its working memory live here.
• VRAM. Memory on a dedicated GPU. For GPU inference this is usually the hard limit, and cards with different memory sizes are very different targets.
• Unified memory. On Apple Silicon, one pool shared between CPU and GPU. It can help larger models fit, but fit is not the same as speed.
• Quantization. A smaller numeric format that reduces memory use, usually with some cost to quality.
• Context length. How much text the model reads at once. Longer context means more tokens held in memory.
• KV cache. The running memory a model keeps during a conversation. It grows with context length.

Loading is only the first hurdle. A model can fit in memory and still be painful to use. How usable it feels depends on memory bandwidth, quantization, context length, the runtime, and everything else the machine is doing at the same time. A model that technically loads may still be too slow for daily work. That is why fit and speed are separate questions, and why we describe fit in plain terms rather than benchmark claims.

Instead of a single pass-or-fail, fit is described with three plain labels, based on your assumptions about the model and how you plan to use it:

• Fits well. Enough memory headroom for the selected model and expected use case.
• Fits with tradeoffs. May work with quantization, shorter context, slower generation, or fewer background tools.
• Not recommended. Likely to be slow, unstable, or too memory-constrained for the model and use case.

These describe fit, not a speed or benchmark result.

A model file on disk does nothing on its own. A model runner is the software that loads a model and serves it as a chat window, a local API, or a command-line tool. LM Studio, Ollama, llama.cpp, and MLX are common examples. A model runner is not an agent runner: it serves the model, while an agent runner manages steps, tools, memory, and state around it.

A general rule can only get you close. The honest answer for your machine comes from checking a specific model against your actual memory. The /run-local tool takes your hardware and tells you whether a given model fits well, fits with tradeoffs, or is not recommended. From there, Builder's Local LLM mode lets you pair the machine with a runner and see the tradeoffs in one place.

If you are still weighing whether local is the right path, compare it with a hosted model in local LLM vs cloud API, and read why local AI is not free before assuming a local setup costs nothing.

• Cost to run an LLM
• Cost to run an AI agent
• Local LLM vs cloud API
• GPU hosting vs model API
• Why local AI is not free
• Browse all stack parts
• Read the FAQ