How ChatGPT works: large language models explained simply
How do chatbots like ChatGPT, Claude and Gemini actually work? A plain-English explanation of large language models: tokens, training, prediction and why they sometimes get things wrong.
First published · AI Digested
When you type a question into ChatGPT, Claude or Gemini, a reply appears a word at a time, often well written and genuinely useful. It's natural to imagine something reading your question, thinking about it and looking up the answer. What actually happens is stranger, and easier to understand than you might expect.
The one-sentence version
A large language model is a giant prediction machine: given some text, it predicts what text is likely to come next. Do that once and you get a word. Repeat it, adding each new word to the text, and you get an answer.
Step 1: Text becomes tokens
Models don't read letters or whole words. They split text into tokens, common chunks of characters. "Chatbots are clever" might become Chat bots are clever. In English, 100 tokens is roughly 75 words.
Each token is turned into a long list of numbers, so the model can do maths with it.
Step 2: Learning from a mountain of text
The model is a neural network with billions of adjustable numbers, called parameters. At the start, they are random, and its predictions are gibberish.
During training, the model is shown text from a huge collection: web pages, books, articles, code and more, adding up to trillions of tokens. Over and over, it sees part of a sentence, guesses the next token, and checks the real answer. Each time, every parameter is nudged slightly so that the right answer becomes a bit more likely.
After this has happened trillions of times, on thousands of specialised chips over several months, the model has become very good at prediction. And here's the key insight: to predict text well, you have to capture a lot about the world.
To predict the next word in "The capital of France is…", it helps to know geography. To continue a maths proof, it helps to follow logic. To finish a line of computer code, it helps to know how programs work. Nobody teaches the model these things directly. They emerge because they make prediction better.
Step 3: Attention, the secret ingredient
Modern language models use a design called the transformer, invented by Google researchers in 2017. Its big idea is attention: when processing each word, the model can look back at every other word in the text and decide which ones matter most.
In "The trophy didn't fit in the suitcase because it was too big", attention helps the model work out that "it" means the trophy, not the suitcase. This ability to connect ideas across long passages is a large part of why today's models are so capable.
Step 4: Turning a predictor into an assistant
A model trained only on internet text is a powerful autocomplete, but not a helpful assistant. Ask it a question and it might just carry on with more questions, because that's what text on the internet sometimes does.
So companies add more training:
- Instruction tuning: the model is trained on many examples of questions with good answers, written by people.
- Reinforcement learning from human feedback (RLHF): people compare pairs of answers and pick the better one. The model is then trained to produce answers people prefer: helpful, accurate, polite, and refusing genuinely harmful requests.
This stage gives each chatbot its personality and its rules.
Step 5: Writing an answer
When you send a message, the app combines it with the conversation so far and some hidden instructions from the company, called a system prompt. The model reads the lot and produces a list of likely next tokens with probabilities. One is chosen, often with a little randomness so replies don't feel robotic. It's added to the text, and the process repeats until the answer is finished.
That's why answers appear word by word: you're watching the predictions happen.
Some newer reasoning models first write out a long, private chain of working before giving you a final answer. Giving the model room to "think" makes it much better at maths, logic and coding.
Why chatbots get things wrong
Because the model generates plausible text rather than retrieving stored facts, it can produce something that sounds right but isn't. It might invent a quote, a statistic or a book title. These mistakes are called hallucinations.
Other limits follow from the same design:
- Knowledge cut-off: the model only knows what was in its training data, unless it's given search tools.
- Maths and counting: working in tokens rather than letters and digits can trip it up, although modern models are much better.
- Context window: it can only consider a certain amount of text at once.
What this means for you
- Treat a chatbot like a very well-read, very fast assistant who is occasionally, confidently wrong.
- Check facts that matter, especially numbers, names, dates, quotes, medical, legal and money questions.
- Give it context and clear instructions. Read how to write good prompts.
Keep going
Common questions
Does ChatGPT search the internet?
Sometimes. The model itself only knows what was in its training data, up to a cut-off date. Many chatbots can now also run web searches and read the results before answering, and they usually show the links when they do.
Does ChatGPT remember my conversations?
Within a conversation, yes: everything so far is fed back into the model each time. Between conversations, only if the app has a memory feature switched on, which saves notes about you separately. The model itself doesn't change when you talk to it.
Why is it called a 'large' language model?
Because of the number of adjustable values, called parameters, inside it, often hundreds of billions, and the vast amount of text it was trained on.
Does it understand what it's saying?
This is genuinely debated. LLMs clearly capture a lot of meaning: they can explain, summarise and reason through problems. But they have no experiences, senses or goals of their own, and they can fail in ways no person who understood the topic would.
Sources and further reading
Spotted a mistake or something out of date? Tell us and we will fix it. Read how we research and write our articles.
Read next
Machine learning explained: how computers learn from examples
What machine learning is, the three main ways machines learn, real examples you use every day, and why data matters more than anything else.
Updated 22 November 2026
What is generative AI? Text, images, video and music made by machines
Generative AI creates new text, images, audio, video and code. Here's how it works, what it's used for, the main tools, and the big arguments about copyright and jobs.
Updated 22 November 2026
What are AI agents? AI that does things, not just says things
AI agents can browse the web, write and run code, fill in forms and complete multi-step tasks for you. What they are, what they can do today, and the risks to understand first.
Updated 22 November 2026