A context window is the total amount of text, measured in tokens, that a large language model can take into account when producing a single response. Everything the model "knows" about a task at that moment has to fit inside it: the instructions, the conversation so far, attached documents, results returned by tools, and the reply being written. Anything outside the window does not exist for the model. The limit exists because the transformer architecture behind every major model compares tokens with one another, and the cost of doing so rises steeply with length.

What fills the window

Anthropic's documentation defines the context window as "all the text a language model can reference when generating a response, including the response itself." That last clause matters. The window is a shared budget for input and output, not an input allowance alone.

A typical request to an AI assistant or agent contains five layers. The system prompt sets the model's role and rules. Tool definitions describe the functions an agent may call, such as a reporting API or a search function. The message history holds every earlier turn. Retrieved material - documents, spreadsheets, web pages, images - is inserted by the application. Finally, the model's output, including any hidden reasoning, is generated into the same space. According to Anthropic, cached prompt tokens, images, documents and tool results all count against the limit, and so does extended thinking on the current turn.

A token averages about four characters of English, so a 1,000,000-token window holds roughly 750,000 words, or several thick novels. A 100,000-token document is, as PPC Land's explainer on tokens puts it, roughly a short book. Tokenizers differ by vendor and language, and the same explainer notes that Anthropic says Claude 4.7 and later models produce roughly 30% more tokens for the same text than earlier versions, which shrinks the effective capacity of a window of the same nominal size.

How the window works

During inference, a model processes the window in two stages. In prefill, it reads every input token and builds an internal representation of each one. In decoding, it generates output one token at a time, with each new token attending to everything before it. Attention compute grows roughly with the square of the input length, and the memory holding intermediate results, called the key-value (KV) cache, grows in proportion to it.

Models also need to know where each token sits. Most modern systems use rotary position embeddings, and a model trained on short sequences cannot reliably handle positions it never saw. Vendors therefore extend windows through additional training on long documents and through position-scaling techniques. Meta's Llama 4 Scout illustrates the gap between training and advertised length: Meta said on April 5, 2025 that Scout offered "an industry leading context window of 10M" tokens, while also stating that it was pre-trained and post-trained with a 256,000-token context length.

What happens at the edge depends on the provider. On Anthropic's API, a request whose input alone exceeds the window is rejected with a "prompt is too long" error. On Claude 4.5 and newer models, a request whose input plus requested output exceeds the window is accepted, and generation stops with the reason "model_context_window_exceeded" when space runs out. Early chat products simply dropped the oldest turns.

Window sizes and how they grew

Early commercial models were small by current standards. OpenAI's GPT-3, presented in May 2020, accepted 2,048 tokens. GPT-4 arrived on March 14, 2023 in 8,192-token and 32,768-token versions, and Anthropic expanded Claude to 100,000 tokens in May 2023. Google's Gemini 1.5 Pro moved the frontier in February 2024. PPC Land's report on the Gemini 1.5 announcement described a 1,000,000-token window being tested with selected users while the wider release used 128,000 tokens.

The million-token tier then spread. Anthropic opened a 1,000,000-token window for Claude Sonnet 4 on August 12, 2025, initially in beta with higher prices for long prompts. As of November 2026, Anthropic's documentation lists a 1,000,000-token window and a 128,000-token maximum output for all four current models - Fable 5.1, Opus 5.5, Sonnet 5.5 and Haiku 5.5 - with no beta header required and standard pricing, except on Haiku 5.5. Google's model page, updated October 28, 2026, describes Gemini 4 Argon as having "a unified 1M context window". OpenAI's model documentation lists windows of about 1.05 million tokens for its current flagship models.

Size and price now move together. Haiku 5.5 costs $0.10 per million input tokens for prompts up to 100,000 tokens and $0.50 above that threshold, a fivefold step. Anthropic said the cheaper tier covered around 90% of requests to the previous Haiku model.

Managing a full window: compaction, editing and caching

Long-running agents fill windows quickly because every step re-sends the accumulated history. OpenRouter data showed AI agents consuming five times more tokens than humans, with more than 85% of that consumption coming from cached prompts as agents reload tool definitions and instructions each turn. Three techniques have emerged to cope.

Compaction replaces older turns with a summary. Anthropic's engineering team, in a September 29, 2025 article by Prithvi Rajasekaran, Ethan Dixon, Carly Ryan and Jeremy Hadfield, described it as "taking a conversation nearing the context window limit, summarizing its contents, and reinitiating a new context window with the summary." Anthropic's API now offers two server-side versions in beta. One triggers automatically at a token threshold, by default 150,000 input tokens with a minimum of 50,000, and wraps the result in summary tags. The other, under a beta header dated September 4, 2026, lets the application decide when to compact. The summarisation call is billed like any other request, and the documentation states there is no option to use a cheaper model for the summary. OpenAI's Responses API offers an equivalent: a compact_threshold setting, such as 200,000 tokens, that triggers server-side compaction, plus a standalone compact endpoint that returns an encrypted compaction item.

Context editing removes material rather than summarising it. Anthropic's tool-result clearing, by default, starts once input reaches 100,000 tokens and keeps the three most recent tool calls, replacing older results with placeholder text. A separate strategy clears old reasoning blocks. Anthropic's article called tool-result clearing "one of the safest lightest touch forms of compaction."

Caching does not enlarge the window but makes refilling it cheaper, at one-tenth of the base input rate for reads on Anthropic's models. Cached tokens still count toward the limit.

Other approaches work outside the window. Retrieval-augmented generation fetches only relevant passages, and agents can write notes to external storage. According to the same Anthropic article, a subagent may use "tens of thousands of tokens or more" exploring a problem but returns a summary "often 1,000-2,000 tokens" long.

Why it matters for marketers

The window decides what an AI tool can actually see. An assistant asked to audit an advertising account, compare a quarter of campaign reports or rewrite 500 product listings works only on what fits. PPC Land's coverage of context engineering in July 2025 noted that general-purpose tools "process limited data subsets due to context window constraints", and set out four strategies - write, select, compress and isolate - for deciding what enters.

For agentic media buying, the window is also a cost line. Each tool call adds results to the history, and each step re-reads it. One ad tech vendor collapsed 12 MCP calls into a single buyer agent to stop the resulting token burn.

Limitations and disputes

The advertised window is not the usable window. Nelson Liu and colleagues at Stanford reported in July 2023 that performance "significantly degrades when models must access relevant information in the middle of long contexts", a pattern known as lost in the middle. On July 14, 2025, Chroma researchers Kelly Hong, Anton Troynikov and Jeff Huber tested 18 models, including GPT-4.1, Claude 4 and Gemini 2.5, and found that "models do not use their context uniformly"; performance fell as input grew, even on simple tasks. They called the effect context rot. Anthropic's documentation now uses the term itself, stating that "as token count grows, accuracy and recall degrade."

Benchmarks are contested. Vendors often cite needle-in-a-haystack tests, which hide one fact in a long document and ask for it back. Chroma's results suggest such tests flatter models, because performance dropped faster when the question and the hidden fact shared less vocabulary and when similar distractors were present. Meta's Scout figure raises a different objection: a window advertised well beyond training length invites questions about how much of it is reliable.

Compaction has its own trade-off: a summary discards detail, and errors made at that step persist in every later turn.

Not the same as

Token limit is the broader family of ceilings. It includes the context window but also the maximum output per reply and the per-minute rate limits set on an account.

Maximum output caps only the reply, usually through a max_tokens parameter. As of November 2026, Anthropic's current models allow 128,000 output tokens inside a 1,000,000-token window.

Memory in chat products, such as saved preferences or past-conversation recall, is stored outside the window and reinserted when relevant. It consumes window space only when loaded.

Recent developments

Recent months brought pricing and tooling rather than larger windows. Anthropic's Haiku 5.5 launch on October 7 set a 100,000-token threshold for long-context rates and positioned the model for compaction and summary work. Google's documentation, updated October 28, lists Gemini 4 Argon with output of up to 1,000,000 tokens. Anthropic's on-demand compaction moved control of summarisation to the application.

Timeline

  • June 2017 - Google researchers publish the transformer architecture, which processes fixed-length token sequences.
  • May 2020 - OpenAI presents GPT-3 with a 2,048-token window.
  • March 14, 2023 - GPT-4 released in 8,192-token and 32,768-token versions.
  • May 2023 - Anthropic expands Claude to 100,000 tokens.
  • July 6, 2023 - "Lost in the Middle" paper submitted, showing weaker recall from the middle of long inputs.
  • February 15, 2024 - Google announces Gemini 1.5 Pro with a 1,000,000-token window for testers.
  • April 5, 2025 - Meta releases Llama 4 Scout with an advertised 10,000,000-token window.
  • July 14, 2025 - Chroma publishes its context rot study of 18 models.
  • August 12, 2025 - Anthropic opens a 1,000,000-token window for Claude Sonnet 4 in beta.
  • September 29, 2025 - Anthropic publishes its context engineering guidance on compaction, note-taking and subagents.
  • September 4, 2026 - Date of Anthropic's beta header for on-demand compaction.
  • October 7, 2026 - Anthropic prices Haiku 5.5 with a 100,000-token long-context threshold.
  • October 28, 2026 - Google's model documentation lists Gemini 4 Argon with a 1,000,000-token window.

Summary

Who. Model developers such as Anthropic, OpenAI, Google and Meta set context window sizes. Developers, agencies, ad tech vendors and marketers building assistants and agents work within them.

What. A context window is the maximum number of tokens a model can consider in a single request, covering instructions, history, documents, tool results and the reply. As of November 2026, frontier models commonly offer about 1,000,000 tokens, with Meta advertising 10,000,000 for Llama 4 Scout.

When. Windows grew from 2,048 tokens in 2020 to 100,000 in 2023 and 1,000,000 from 2024. Context rot was documented in July 2025, and server-side compaction APIs followed in 2026.

Where. The limit applies inside model APIs and every product built on them, from chat assistants to campaign copilots and agentic buying systems.

Why. Attention compute and memory grow steeply with length, so windows are finite and costly to fill. For marketers, the window determines how much account data an AI tool can consider at once, and research shows that accuracy declines as it fills.