How does a language model work? In plain words

Short answer: a large language model (LLM) keeps predicting which bit of text is likely to come next. It learned to do that from huge amounts of text. The answer you get is a string of those predictions, one after another.

Text in small pieces: tokens

A language model does not read words the way we do. It cuts text into tokens: whole words, parts of words, punctuation or single characters. According to OpenAI, one token in English is on average about four characters, or three quarters of a word.

Learning in two steps

1. Pre-training. The model sees an enormous amount of text and learns to predict the next token, over and over. That is how it picks up language, facts and patterns. The underlying design is usually the Transformer, described in a 2017 paper.
2. Tuning. The model is then refined into a helpful assistant. When OpenAI launched ChatGPT, it explained how: people wrote example conversations, then ranked the model's answers from good to bad, and the model learned from those rankings. That method is called reinforcement learning from human feedback (RLHF). Further training on extra examples is called fine-tuning.

Why it sometimes makes things up

A language model does not look things up in a database of facts. It writes text that sounds likely. So it can give an answer that reads convincingly and is still wrong. This is called hallucination.

When OpenAI launched ChatGPT, it said so itself: the model sometimes writes "plausible-sounding but incorrect or nonsensical answers", and a small change in your question can change the answer. The Dutch Data Protection Authority lists the risk of incorrect or unwanted answers as a standing point of attention for organisations that use generative AI.

What to do about it

• Check facts, figures and names before a text goes to a customer.
• Give the model your own sources (a document, a price list) and ask it to stick to them.
• Ask for sources and click them. A source that does not exist tells you enough.
• Use AI for the first draft, not the final word.

Sources

• OpenAI Help Center: What are tokens and how to count them?
• OpenAI: Introducing ChatGPT (methods and limitations)
• Vaswani et al., Attention Is All You Need (arXiv, 2017)
• Autoriteit Persoonsgegevens: Hulpmiddel inzet generatieve AI & de AVG (PDF, Dutch)

Accessed 21 October 2026.