Have you ever been deep in a long chat with an AI assistant, only to find it has forgotten something you said at the start? Or pasted in a long report and received an answer that ignored half of it? These frustrations usually trace back to one idea: the context window.
This guide explains what a context window is in plain language, how it is measured, why it affects the quality of your results, and what you can do about it. You will learn how to work within the limits, how to structure long conversations and documents, and how to choose between a larger window and better organisation.
Key takeaways
- The context window is the amount of text an AI model can consider at one time, covering your instructions, the conversation so far, any pasted material and the answer itself.
- It is measured in tokens, which are small chunks of text, not in pages or words, so the exact fit varies.
- When a conversation outgrows the window, older material may be dropped or summarised, which is why tools seem to forget.
- A bigger window does not guarantee better use of the material, because models can still miss details in long inputs.
- The best fix is good organisation: shorter chats, clear summaries, key facts repeated near the request and one task at a time.
What a context window is
An AI language model does not remember things the way a person does. Each time it produces a reply, it reads a block of text and predicts what should come next. The context window is the maximum size of that block. Everything the model can use to answer you has to fit inside it.
A desk analogy
Imagine the model is working at a desk. The desk surface is the context window. Your instructions, the earlier messages and any documents you provide are all papers laid out on the desk. The model can only work with what is on the desk. If the desk is full and you add more paper, something has to be moved off. The model has general knowledge from its training, which is like things it studied years ago, but anything specific to your task has to be on the desk now.
Why it exists
Reading text costs computing power, and the cost grows as the amount of text grows. Every model is built with a limit on how much it can handle at once. If you want a fuller picture of how these models work, our explainer on what a large language model is covers the basics that make the window easier to understand.
It is a limit on attention, not on knowledge
The window does not limit what the model knows in general. It limits what it can look at in this particular exchange. A model with a small window can still know a great deal about history or grammar; it just cannot hold your fifty-page report in view at once.
Tokens: how the window is measured
Windows are measured in tokens rather than words. A token is a chunk of text that the model treats as a unit. It might be a whole short word, part of a longer word, a punctuation mark or a space with a letter.
Why tokens and not words
Models break text into pieces based on patterns learned during their building. Common words are often one token. Rare words, names, technical terms and text in other languages are often split into several. Numbers, code and unusual formatting can also use more tokens than you expect.
A rough rule of thumb
As a loose guide for ordinary English prose, a token is usually shorter than a word, so a given number of words takes somewhat more tokens than words. The exact ratio depends on the tool and the text, so treat any conversion as an estimate. If precision matters, look for a token counter in your tool or in its documentation.
| Type of text | Token use compared with plain English | Why |
|---|---|---|
| Everyday prose | Baseline | Mostly common words |
| Technical or legal text | Often higher | Long, rare terms split into pieces |
| Code | Often higher | Symbols, indentation and unusual names |
| Other languages | Can be higher | Tokenisation is often tuned for the most common languages |
| Tables and formatted data | Often higher | Many separators and repeated structure |
Input and output share the space
The window is not only for what you send. The answer the model writes also takes up room. If you paste in a text that nearly fills the window and ask for a long reply, the reply may be cut short or refused. Leave space for the response.
What takes up space in the window
People often think of the window as holding just their latest message. In reality it contains several things at once.
- Hidden instructions. Many tools add their own background instructions before your message, which use some of the space without you seeing them.
- Your custom settings. Saved preferences or role descriptions you have added may be included each time.
- The conversation history. Every earlier message, yours and the model’s, can count, which is why long chats fill up.
- Attached or pasted material. Documents, tables, emails and code you supply.
- Retrieved material. If the tool searches the web or your files, the results it fetches are added too.
- The answer being written. The reply is built inside the same window.
Why long answers eat space
If you ask for very detailed answers each turn, the conversation grows quickly, because each long reply becomes part of the history. A chat of many long exchanges can reach the limit sooner than one of many short ones.
Why uploaded files can use more than you expect
A file that looks small can use many tokens if it is full of tables, repeated headers or scanned text that has been converted badly. If you notice odd behaviour after an upload, check whether the file converted cleanly and whether it is larger than you thought.
Why AI seems to forget
When the total content exceeds the window, the tool has to decide what to do. Different tools handle this differently, and many do not tell you which approach they use.
| Approach | What happens | What you notice |
|---|---|---|
| Drop the oldest messages | Earliest content is removed first | It forgets your original instructions |
| Summarise older content | The history is compressed into a short recap | Details and nuance disappear |
| Refuse or error | The tool says the input is too long | You must shorten or split it |
| Truncate silently | Part of your input is cut off | The answer ignores the end of your document |
The vanishing instruction problem
The most common form of forgetting is losing your first instruction. You began the chat by saying “always answer in plain English and keep it under 100 words”. Forty messages later, replies are long and technical. The instruction has scrolled out of the window or been compressed away. The fix is to restate key rules when they matter.
Forgetting is not lying
When the model has lost earlier context, it does not know that it has. It will often carry on confidently, filling gaps with plausible guesses. This is one route to the invented details described in our guide to AI hallucinations. If an answer contradicts something you said earlier, suspect a context problem before suspecting stubbornness.
Bigger windows are not a cure-all
It is tempting to think that a larger window solves everything: just paste in the whole archive and ask your question. In practice it is more complicated.
Attention is uneven
Models can give more weight to some parts of a long input than others. Details buried in the middle of a very long text are sometimes handled less reliably than details near the start or end. Different models behave differently, and this changes over time, so test rather than assume.
More material means more noise
If you include a lot of irrelevant text, the model has more to sift through, and the answer can drift toward whatever was most prominent rather than what mattered. A short, focused input often beats a huge one that contains the answer somewhere inside.
Cost and speed
Processing more text generally takes longer and may use more of your usage allowance. Large inputs can slow replies and may hit plan limits sooner. Check how your plan counts usage; our guide to reading an AI tool’s pricing page explains what to look for.
When a large window really helps
- Comparing several long documents against each other
- Asking questions across a whole report, contract or codebase
- Keeping a long working session coherent without constant recaps
- Summarising material that cannot easily be split
How to work within the limits
Whatever the size of your window, the same habits make results better. These are the practical steps.
- Put the task first, or repeat it at the end. State clearly what you want, then give the material, then restate the question if the material is long.
- Include only what is relevant. Cut background that the model does not need. Remove signatures, repeated headers and unrelated sections.
- Label your material. Use plain markers such as START OF REPORT and END OF REPORT so the model knows where each item begins and ends.
- Restate key rules. If an instruction matters, repeat it when you make the request instead of relying on something said long ago.
- Ask for what you need, not everything. Specify the sections or questions you care about rather than requesting a total summary.
- Leave room for the reply. Do not fill the window to the brim.
Write prompts that survive truncation
Assume that anything might be cut. Put the most important constraints in the message you are sending now. For tips on structuring requests, see our guide to writing better prompts.
Use structure
Headings, numbered lists and clear labels help the model find what it needs in a long input. A wall of unbroken text is harder for anyone to navigate, human or machine.
Working with long documents
Documents are where most people hit the limit. Here are reliable ways to handle them.
Split and summarise
Divide the document into logical sections, such as chapters or topics. Ask for a summary of each section, then give the summaries to the model and ask your final question. This is slower, but it keeps each step within the window and lets you check the intermediate summaries for errors.
Ask targeted questions
Instead of “summarise this report”, ask “what does the report say about delivery delays, and on which pages or sections?” Targeted questions with a request to point to the location make it easier to verify the answer.
Ask for quotes
Ask the model to quote the exact sentence that supports each claim. Then check those quotes against the original. If a quote does not appear, you have caught a mistake before it spread. Our guide to checking AI output for mistakes describes more checks of this kind.
Compare approaches
| Approach | Best for | Drawback |
|---|---|---|
| Paste the whole document | Short or medium texts that fit comfortably | Risk of missed details in long inputs |
| Split into sections | Long reports with clear structure | More steps and more checking |
| Summary then question | Overview tasks | Summaries may drop nuance |
| Targeted extracts | Specific questions | You must find the right extract first |
| Tools that search your files | Large collections of documents | Quality depends on what the search retrieves |
Managing long conversations
A single chat that runs for days is convenient, but it is also where context problems grow.
Start fresh when the topic changes
Unrelated earlier content can confuse the model and waste space. When you move to a new task, open a new conversation. Carry over only what is needed.
Write a handover summary
When a chat gets long, ask the model to write a short summary of decisions, facts and open questions. Check it, correct it, and paste it as the first message of a new chat. This keeps the good parts and drops the clutter.
Keep a pinned brief
Maintain a short document with your standing instructions: audience, tone, definitions, things to avoid. Paste it at the start of each new chat. It takes seconds and avoids repeating yourself. This is the same idea as the reusable pieces in our work on prompt templates.
Spot the warning signs
- The model repeats a question you already answered
- It forgets a name, number or rule you gave earlier
- Its style drifts from what you asked for
- It contradicts its own earlier statements
- Replies become vague or generic
When you see these, do not argue. Summarise, restate the key rules and start a fresh chat.
Context window versus memory
Some tools advertise memory, which can be confused with the context window. They are related but different.
What memory features usually are
A memory feature typically stores short notes about you or your preferences outside the conversation and adds relevant notes to the window when needed. It is a way of getting information into the window, not a separate unlimited store inside the model.
What this means for you
- Stored notes can be wrong or out of date, so review them if the tool lets you.
- Notes take up window space when added, so they are still competing with everything else.
- Anything stored may be kept by the provider, so think before you let a tool remember sensitive details. Our guide to protecting company data when using AI tools is a good companion read.
Tools that act on your behalf
AI agents often work in many steps, accumulating context as they go. Long chains can run into the same limits and drift off task. If you are curious how this works, see our plain-language explainer on AI agents.
What to look for when choosing a tool
Window size is one factor among many. Use it as a prompt for sensible questions.
| Question | Why it matters |
|---|---|
| Does the tool state its window size or file limits? | Clear limits help you plan your work |
| What happens when I exceed the limit? | Silent truncation is riskier than a clear warning |
| Can I see how much of the window I have used? | Visibility helps you manage long chats |
| Are limits different on free and paid plans? | You may be testing a smaller window than you will pay for |
| Does it cite where in a document an answer came from? | Citations make checking easier |
When you compare tools, test them with your own long documents, not demo files. Our guide to choosing an AI tool sets out a method for doing this fairly.
Context checklist
Before you start a task
- Is this a new topic that deserves a fresh chat?
- Have I removed irrelevant material from what I plan to paste?
- Have I stated the task and the format I want?
- Have I left room for a long reply?
While you work
- Are key rules restated when they matter?
- Am I asking targeted questions instead of broad ones?
- Have I asked for quotes or locations to check claims?
- Am I watching for signs of forgetting?
When the chat gets long
- Ask for a summary of decisions and open questions
- Correct the summary yourself
- Start a new chat and paste the summary and your standing brief
- Check the first few answers against what you know
Frequently asked questions
Is the context window the same as the model’s memory?
Not really. The window is what the model can see right now in this exchange. Memory features are extra tools that store notes and feed them into the window. The model itself does not keep your conversation after it ends unless the tool is built to save it.
How can I tell how many tokens my text uses?
Some tools show a count or a usage meter, and some providers publish token counting tools. If none is available, use a cautious estimate and leave a generous margin. Remember that code, tables and non-English text can use more tokens than plain prose.
Why did the AI ignore part of my document?
Possible reasons include the document being cut off because it was too long, the details being buried in a very long input, or the question being too broad. Try splitting the document, asking a targeted question and requesting quotes for each claim.
Does a bigger window mean a better model?
No. Window size says how much text can fit, not how well the model understands or reasons about it. A model with a large window can still make errors, and a model with a smaller window can be excellent at focused tasks.
Should I paste everything just in case?
Usually not. Extra material adds noise, uses allowance and may reduce accuracy. Include what is relevant to the question, and keep the rest ready in case you need it for a follow-up.
Can I extend the window myself?
You cannot change the model’s limit, but you can get more from it by summarising, splitting tasks and starting clean chats. Some tools also search your files and add only the relevant parts, which helps with large collections.
Conclusion
The context window is the working space an AI model has for each reply. It is measured in tokens, it is shared between your input and the answer, and it explains much of the forgetfulness and drift people experience in long chats. A larger window can help, but organisation matters more than size: focused inputs, restated rules and tidy conversations beat a giant paste every time.
Put the habits in this guide to work on your next long task. Split the document, ask targeted questions and keep a short brief you can reuse. If you want to go further, our guide to writing better prompts helps you phrase requests clearly, and what is a large language model explains the engine underneath all of this.
