A context window is the maximum amount of text (measured in tokens) an AI model can process at one time, including your prompt, chat history, and any documents. Exceed it and the model forgets or truncates earlier information.
Why it matters
If your context window is too small, the model forgets earlier instructions, drops details from long documents, or loses track of a long conversation. This directly affects tasks like analyzing contracts, summarizing customer chats, or maintaining a long prompt chain.
Tokens, not words
Context windows are measured in tokens, not words or characters. Roughly, 1,000 tokens equals about 750 English words, so a 128,000-token window holds roughly 96,000 words, still less than a long business report plus chat history.
The business tradeoff
Bigger context windows cost more to run and can slow responses, so vendors balance size against price and speed. For commerce and ops use cases, look at whether your provider counts the system prompt, retrieved documents, and chat history all against the same limit.
Workarounds teams use
Instead of stuffing everything into one window, many teams use retrieval-augmented generation to pull in only the relevant snippets, or summarize older parts of a conversation to keep it within budget.
Frequently asked
Is a bigger context window always better?
Not necessarily; larger windows cost more and can slow down responses, and models don't always use extra space effectively.
Does the context window include my previous messages?
Yes, in a chat, prior turns count toward the same limit as your current prompt and any attached documents.
What happens if I go over the limit?
The model typically truncates or drops the oldest content, which can cause it to 'forget' earlier instructions or context.