How LLMs Actually Work
The five-minute version: next-token prediction, attention, context, sampling, and why the model sounds smarter than the mechanism.
An LLM generates text one token at a time. That is the plain mechanism. The shocking part is how much useful behavior falls out of doing that at enormous scale.
Tokens
Models do not read text exactly like humans read words. They read tokens: chunks of text that can be whole words, parts of words, punctuation, or weird fragments. Pricing, context limits, and speed are usually measured in tokens.
Training
During training, the model sees huge amounts of text and learns to predict the next token. It does not store a database of every page. It learns statistical structure: grammar, facts, formats, code patterns, reasoning traces, and plenty of internet nonsense.
Attention
Attention lets the model decide which parts of the current context matter for the next token. This is why earlier instructions, examples, function schemas, retrieved documents, and chat history can influence the answer.
Sampling
At generation time, the model produces probabilities for possible next tokens. The runtime picks from them using settings like temperature and top-p. Lower randomness tends to be more stable. Higher randomness can be more creative and more chaotic.
Why it feels like thinking
The model is not running a normal program internally. It is generating plausible continuations from learned patterns and current context. Tool use, retrieval, and evals matter because they wrap that generator in systems that can fetch facts, take actions, and check results.