Guide

What is a model runner?

A model runner is the software that loads a model file and serves inference. It takes the model on disk, puts it in memory, and answers requests. LM Studio, Ollama, llama.cpp, MLX, and vLLM are all model runners. The runner is not the model, and it is not the agent. Picking a runner is a separate choice from picking a model or an interface, and the runner you choose sets the memory floor for what will fit on your hardware.

What a model runner does

The runner is the piece that turns a model file into something you can actually talk to. It handles the plumbing between the model and everything else:

Loads the model. It reads the model file, applies the quantization it was saved with, and puts the weights into RAM, VRAM, or unified memory.
Serves inference. It takes a prompt, runs it through the model, and streams back a reply.
Uses your hardware. It decides how much of the work runs on the GPU versus the CPU, which is why the runner affects whether a model fits at all.
Exposes a way to call it. That might be a chat window, a command-line tool, or a local API that other apps can send requests to.

What a model runner does not do

A runner is often confused with the other parts of a stack. It helps to be clear about what it is not:

It is not the model. The model is the file with the trained weights. The runner is the program that loads it. The same runner can load many different models, and the same model can be loaded by different runners.
It is not always the interface. Some runners ship with a chat window, but the interface is the thing you look at and type into. A runner can serve a local API with no window of its own, and a separate interface can point at it.
It is not the agent runner. A Model runner serves text from a model. An Agent runner manages the loop that calls the model and tools. More on that distinction below.

Common model runners

These are the tools you will run into most often. They overlap, and the right one depends on whether you want a simple local chat or a server that other apps call:

LM Studio. An approachable desktop app that bundles a model runner with a chat window, so you can download a model and talk to it without touching a command line.
Ollama. A local runner with a command-line tool and a local API, which makes it a common choice when you want other apps to call the model.
llama.cpp. The lower-level engine that many other runners build on. It gives you the most control and the least hand-holding.
MLX. Apple’s framework for running models on Apple silicon, which uses the machine’s unified memory.
vLLM. A serving-oriented runner aimed at running an open model as a server for many requests, rather than a personal desktop chat.

If you are weighing the two most common desktop options, see LM Studio vs Ollama.

Desktop runners vs server runners

A rough split helps: desktop runners are usually the better fit for personal local chat, where you want a window and little setup. Server runners are usually the better fit when other apps or users need to call the model over an API. This is a tradeoff, not a ranking. A desktop runner can still expose a local API, and a server runner can still be run on one machine for one person. Assume your usage pattern decides which side you land on.

Model runner vs interface

The Interface is what you see and type into; the Model runner is what loads the model and answers. They are often bundled (LM Studio is a runner and a chat window in one), but they are still separate roles. Keeping them apart matters because you can swap the interface without changing the runner, or point one interface at a runner on a different machine. Picking a runner is a separate decision from picking the interface you want to use.

Model runner vs agent runner

This is the distinction that trips people up most. The two do different jobs and sit at different levels:

Model runner: runs the model. It loads the model file and serves inference. Ask it for text, and it gives you text.
Agent runner: runs the loop. It manages the work around the model: it decides the next step, calls tools, keeps state and memory, retries failures, and gives a human a way to inspect what happened.

An Agent runner sits above a Model runner and calls it. You can have a Model runner with no agent at all. That is local chat. For the other half of the picture, see what is an agent runner?

How AIStackPicker uses runner compatibility

AIStackPicker treats the runner as its own part of the stack, separate from the model and the interface. Because the runner sets the memory floor, it is a big factor in whether a model fits on your Local hardware. The same model can fit under one runner and not another. When you plan a local setup in the Builder, the runner is part of the fit check, not an afterthought. If you are unsure your machine can handle it, start with can my computer run a local LLM?

FAQ

Is a model runner the same as the model?

No. The model is the file with the trained weights. The Model runner is the software that loads that file and serves inference. One runner can load many models, and one model can be loaded by different runners.

What is the difference between a model runner and an agent runner?

A Model runner loads and serves the model. An Agent runner manages the loop around it: steps, tool calls, memory, retries, and state. The Agent runner sits above the Model runner and calls it.

Does the runner I pick affect whether a model fits?

Yes. The runner sets the memory floor and decides how work is split between GPU and CPU, so the same model can fit under one runner and not another on the same Local hardware. Assume fit depends on both.