Running Local Models: The Software Guide
Ollama, LM Studio, Jan, llama.cpp, vLLM, and which one to open before you lose an afternoon.
Local model software falls into a few practical buckets.
Start here
Ollama is the easiest developer default. It gives you commands, a local API, and a simple mental model: pull model, run model.
LM Studio is the easiest GUI default. It helps you browse, download, and chat with models without living in the terminal.
Jan is a local chat app for people who want a desktop-style experience.
Go lower-level
llama.cpp is the engine layer. You use it directly when you want control or when another tool is not doing what you need.
koboldcpp is popular in some local chat and roleplay communities, especially where specific UI and generation controls matter.
Production-ish
vLLM is for serving models efficiently, usually with GPUs. If you are thinking about throughput, batching, and an API for other services, you are closer to vLLM territory than LM Studio territory.
The rule
Use Ollama to learn and integrate. Use LM Studio to explore. Use vLLM when local becomes infrastructure.