What is a context window? AI's working memory explained
A context window is how much text an AI model can consider at once. Learn what counts toward it, why long chats go wrong, and how to get better answers.
First published · AI Digested
A context window is the amount of text an AI model can look at when it writes a reply. Think of it as the model's working memory for the current conversation: everything inside it can influence the answer, and anything outside it, as far as the model is concerned, does not exist.
What counts toward the window
The window is measured in tokens, the small chunks of text models read and write. As a rough guide, 100 tokens is about 75 English words. Anthropic's documentation says the context window covers all the text a model can reference when generating a response, including the response itself. In practice that means:
- the instructions the app gives the model behind the scenes (the "system prompt");
- every message you have sent and every reply so far;
- documents, web pages, images or PDFs you have attached;
- results from tools, such as a web search the model has run;
- the answer being written, including any hidden "thinking" for models that reason before replying.
Each time you send a message, the app usually sends the whole conversation back to the model again. That is why long chats take up more and more of the window.
It is not the same as training or memory
Two things are easy to confuse with the context window.
- Training data is the vast text the model learned from before release. It is baked into the model and doesn't change during your chat. Anthropic's documentation says the window is different from this large body of training data.
- Memory features are optional extras in some apps that save notes about you between conversations. They work by putting saved notes back into the context window at the start of a chat. See our guide to AI and your privacy for how to manage them.
How big are context windows?
They have grown enormously. Early chatbots handled a few thousand tokens, a few pages. Anthropic says several of its current models have a window of up to 1 million tokens, which by OpenAI's rule of thumb is roughly 750,000 words, more than several novels. Sizes differ between models and change often, so check the provider's documentation for the one you use.
Bigger is not automatically better
A huge window sounds like a cure for forgetting, but there are catches. Anthropic's documentation is blunt: "more context isn't automatically better. As token count grows, accuracy and recall degrade, a phenomenon known as context rot." It adds that choosing what goes into the context matters as much as how much room there is.
Long contexts can also be slower and, for developers paying per token, more expensive. And models can still make things up about a long document, so check important details against the original.
What happens when the window fills up
Different tools handle it differently:
- Chat apps may quietly drop the oldest messages, or summarise earlier parts of the chat.
- Developer tools may refuse a request that is too long. Anthropic's documentation says the API returns an error if the input alone exceeds the window.
- Some systems "compact" a conversation automatically by summarising it, which saves space but loses detail.
What it means for you
- Start a new chat for a new task. Old, unrelated messages take up space and can distract the model.
- Put the important information first, and repeat it if the conversation is long.
- Don't paste more than you need. If you want a summary of a long report, give the relevant sections.
- Check the answer against the source when the document is long or the stakes are high.
- Don't assume it remembers. If you need the AI to know something tomorrow, paste it in again or use a memory feature deliberately.
Read on
Common questions
Does the AI remember my earlier chats?
Not through the context window. It only covers the current conversation. Some apps add separate memory features that save notes between chats, which is a different thing and can usually be switched off.
Why does the AI forget things I said earlier?
If a conversation gets very long, an app may drop or summarise the oldest messages to make room, and even inside the window a model can pay less attention to details buried in the middle. Restating key facts helps.
Is a bigger context window always better?
No. It lets you include more, but more text can mean slower, costlier replies, and Anthropic's documentation notes that accuracy and recall can fall as the amount of context grows, a problem it calls context rot.
Sources and further reading
Spotted a mistake or something out of date? Tell us and we will fix it. Read how we research and write our articles.
Read next
What is a token in AI, and why are you charged per token?
Tokens are the small chunks of text AI models read and write. Learn how they work, how to estimate them, and why AI companies charge and set limits by token.
Updated 22 November 2026
How ChatGPT works: large language models explained simply
How do chatbots like ChatGPT, Claude and Gemini actually work? A plain-English explanation of large language models: tokens, training, prediction and why they sometimes get things wrong.
Updated 22 November 2026
What is artificial intelligence? A plain-English explanation
What AI actually is, how it learns, what it can and can't do, and why it suddenly seems to be everywhere. No jargon, no maths.
Updated 22 November 2026