Guide

How much does it cost to run an LLM?

The cost to run an LLM depends less on the model name and more on how you run it. A light API workload might be a few dollars a month. A rented GPU left on all month can be hundreds. Local hardware you already own has no token bill, but it is not free. You still pay in hardware, power, and setup. The reason the range is so wide is that these are three different cost models, not three prices for the same thing.

What changes the cost

Before the dollar figure, the shape of the bill is set by how you run the model:

How you run it. An API charges by usage; a rented GPU charges by time; local hardware is paid up front. Same model, different bill.
Usage. On an API, more input and output tokens cost more. Longer context means more tokens processed, so it costs more.
Model size and tier. A larger or higher-quality model costs more per token, and needs more memory to run locally.
Usage pattern. Occasional, daily, and always-on are very different. A rented GPU left on overnight still bills for the idle hours.
The extras. Hosting, storage, and monitoring sit on top of the model cost, especially once something needs to stay running.

The three main ways to run an LLM

A model API charges by tokens. A rented GPU charges by time. These are two different meters, and local hardware is a third. Comparing them means comparing the meters, not only the sticker price.

API model: pay by tokens. You call a hosted model and pay for the input and output it processes. There is no server to keep running, so an idle month costs close to nothing.
Cloud GPU (rented GPU): pay by time. You rent a machine by the hour to run an open model yourself. A rented GPU charges by time, not tokens: whether it answers one request or a thousand, the hour costs the same. Monthly equivalent means hourly price × 730 hours, so a GPU left on all month is the hourly rate multiplied out, often more than an API for light use.
Local hardware: pay up front. You run the model on a machine you own. Local inference has no token bill, but it is not free: you have already paid for the hardware, you pay for power while it runs, and you spend setup time getting the Model runner working and the model to fit in memory.

Example scenarios

These are illustrative assumptions, not quotes. Your usage and the model you pick move the numbers a lot:

A developer asking a model a few times a day is usually served well by an API. Light token use often lands at a few dollars a month, and nothing runs when they are away.
A team serving an open model all day may need to compare a rented GPU against API usage. A GPU left on all month can run into the hundreds, so it only pays off if it is kept busy.
A local user with capable hardware may have no token bill at all, but still accounts for whether the model fits in memory and the power the machine draws while it works.

When each one makes sense

API pricing tends to fit low or bursty usage and cases where you want a frontier-quality model without owning hardware. You pay only for what you send.
A rented GPU or self-hosting can make sense at high, steady usage, where the hourly rate is spread across a lot of work, but it adds setup, maintenance, and monitoring that an API does not.
Local hardware can make sense for privacy, offline work, or repeated tasks on a machine you already own and keep on when you need it.

Common mistakes

Comparing a GPU's monthly equivalent to API pricing without utilization. A GPU that sits idle still bills for every hour. The monthly-equivalent figure assumes the machine is on all month, so only compare it to an API once you know how busy it will be.
Assuming self-hosting is always cheaper. It can be cheaper at high utilization, but it adds setup, maintenance, reliability, and monitoring work that does not show up in the token price.
Treating token pricing and hourly pricing as the same meter. Tokens scale with what you send; hours scale with wall-clock time. Mixing them up is how a cost estimate goes wrong.

How AIStackPicker estimates it

AIStackPicker keeps the three cost models separate instead of blending them into one number. Token-based API usage is estimated from your assumed usage; a rented GPU is converted to a monthly equivalent at hourly price × 730 hours; local hardware is treated as an up-front machine plus running cost, not a token bill. Every price carries a source and the date it was last checked, so you can see how current each figure is before it drives a recommendation.

FAQ

Is running an LLM locally free?

Local inference has no token bill, but it is not free. You have already paid for the hardware, you pay for power while it runs, and you spend setup time getting it working and fitting the model in memory.

Why is a rented GPU sometimes more than an API?

A rented GPU charges by time, not tokens. Left on all month, its monthly equivalent is the hourly price × 730 hours whether it is busy or idle, so for light use it can cost more than paying an API per token.

Does a longer prompt cost more?

On an API, yes. Longer context means more tokens are processed, so both the input you send and the reply you get back can raise the cost of a single call.