A language model doesn’t write. It predicts the next token, one at a time, from a probability distribution over its whole vocabulary. The single knob that most changes the “personality” of the output is temperature.
Drag the slider. The bars are the model’s confidence in each candidate next word after “The animal was a ___“.
- Low temperature (→ 0): the distribution sharpens. The top choice dominates, so the model becomes focused, factual, and repetitive. Great for code and extraction.
- High temperature (→ 2): the distribution flattens. Unlikely words get a real chance, so the model becomes surprising, creative, and occasionally unhinged.
Where the numbers come from
The model outputs a raw score (a logit) for every token. Temperature divides every logit before the softmax squashes them into probabilities:
Divide by a small T and differences get exaggerated (peaky). Divide by a large T and everything evens out (flat). Temperature never adds knowledge, it only reshapes the confidence the model already has.
Practical notes
- Pair temperature with top-p (nucleus) sampling to cut off the long tail of nonsense while keeping variety.
- “The model made something up” is often a sampling story as much as a hallucination story: a confident-but-wrong distribution, sampled boldly.
This is the last step of generation. To close the loop, see how the model got those probabilities in the first place.