How large language models actually work
One capability explains everything an LLM does: it predicts the next token. This is a plain-English, hands-on walk through the machine underneath GPT, Gemini and Claude, written for people who ship with AI rather than train it.
An LLM reads your input, converts it to numeric tokens, passes those tokens through many stacked layers of transformer calculations, and outputs the most statistically likely next token. Then it does that again, and again, until the response is complete.
The result is not a database of facts. It is a dense statistical map of how language is structured, which concepts sit near each other, and which responses people found useful. Understanding that one sentence is most of the way to understanding why LLMs are so capable, and exactly where they break.
Four steps, every single time you hit send
These run in strict sequence for every token the model produces. Tap a step to open it up.
The model never sees your words
It sees integers. Text is split into subword fragments (tokens), and each fragment maps to a number. Type anything and watch it happen.
From numbers to meaning, then to context
Each token ID becomes an embedding, a long vector that places it in a semantic space where "doctor", "physician" and "surgeon" land close together. Positional encoding is added so "the cat sat on the mat" is not confused with "the mat sat on the cat".
Then comes the transformer. Older networks read one word at a time, so linking a pronoun in sentence ten back to a noun in sentence two was hard. The transformer processes every token in parallel, and every token attends to every other token at once through self-attention. For each token the model builds three vectors:
Query
What this token is looking for in the sequence.
Key
What this token offers to the others.
Value
The information this token contributes if attended to.
The model scores each query against every key, turns those scores into attention weights, and blends the values accordingly. A frontier model runs dozens of attention heads across dozens of layers. One head might track subject-verb agreement, another coreference, another position. Attention handles structure and context. The feedforward layers that follow are where most factual knowledge lives.
Temperature is the precision dial
After the layers run, the model has a probability for every possible next token. Temperature decides how it picks. Slide it, watch the distribution reshape, then generate a token.
Three stages of training
Pre-training builds a model that can continue text. The next two stages make it a useful assistant.
Pre-training
Predict the next token across trillions of tokens from web text, books, code and papers. Every wrong guess adjusts hundreds of billions of weights. Weeks to months on tens of thousands of GPUs.
Supervised fine-tuning
Human trainers write example conversations. The model learns the instruction-following format from thousands of them.
RLHF
Human raters compare pairs of answers. A reward model learns their preferences, and the LLM optimises for outputs that score well. This is what turns raw text prediction into genuinely helpful behaviour.
Traditional AI versus large language models
The shift from many narrow models to one foundation model changed the economics of building with AI. Prompt engineering became the configuration layer.
| Traditional AI | Large language models | |
|---|---|---|
| Training | Task-specific, labelled datasets | General pre-training on web-scale data |
| Flexibility | One model per task | One foundation model, many tasks |
| Architecture | SVM, LSTM, CNN, rule-based | Transformer, decoder or encoder-decoder |
| Scale | Millions of parameters | Billions to trillions of parameters |
| Instructions | Structured input, fixed output | Natural language instructions |
| Knowledge update | Retrain on new data | Retrieval or fine-tuning |
Can an LLM own this task?
Reliability tracks one thing: how many similar examples sat in the training data, and how clearly you can check the output. Pick a task.
Three failure modes that are structural
These are not prompt bugs. They come from how the machine is built, so you design around them rather than fixing them away.
Hallucination
Confident, plausible, and wrong. The model predicts what fits the pattern of an authoritative answer even when it does not know.
Knowledge cutoff
Training data ends on a fixed date. Anything after it is unknown unless you fetch it in at query time.
Context confusion
In long threads, early instructions lose influence as the window fills. The model leans on recent tokens.
What actually matters when you build with this
Less about the mathematics, more about the constraints that shape your decisions.
Context window is working memory
Everything inside is visible, everything outside is gone. In your architecture it is a real cost line, not a footnote.
Hallucination is structural
There is no fact-checker at inference. Decide where human review is mandatory before you ship, not after.
Prompt before you fine-tune
A few good examples get most of the benefit at a fraction of the cost. Fine-tune only when you need consistent format at scale.
Temperature is a dial you own
0 to 0.2 for extraction and classification, higher for ideation. It is a product decision, not a default.