Retrieval-augmented generation, usually shortened to RAG, is a method for making a large language model (LLM) look something up before it answers. Rather than relying only on knowledge compressed into its weights during training, the system searches an external collection of documents, inserts the most relevant passages into the prompt and asks the model to write its response from them. The technique exists because trained models are frozen at a cutoff date, cannot say where a fact came from and produce plausible text when they do not know the answer. It now sits beneath AI Overviews, ChatGPT search, Perplexity and a growing number of advertising agents.

How the pipeline works

A RAG system runs in two phases. The first, indexing, happens offline. A corpus is assembled: a product catalogue, a help centre, a contract archive or a crawl of the public web. Documents are cut into chunks, usually no more than a few hundred tokens each, according to Anthropic's engineering documentation. Each chunk passes through an embedding model, which converts text into a vector, a long list of numbers that positions the passage in a mathematical space where similar meanings sit close together. The vectors are stored in a vector database next to the original text.

The second phase runs at query time. The user's question is embedded with the same model, and the database returns the chunks whose vectors lie nearest. Many production systems run a keyword method such as BM25 in parallel, because semantic search alone misses exact strings like product codes, and merge the two lists. A reranker, a second model that scores each candidate against the question, then trims the set. The surviving passages are inserted into the prompt with instructions to answer from them and cite them. Generation follows. The model itself is unchanged; only its input differs.

The complaint CNN filed against Perplexity on May 28, 2026 compressed the flow into four steps: receive a prompt, retrieve related content, combine the two, and pass the bundle to a model. Google's documentation for EmbeddingGemma, released in September 2025, stated plainly that RAG quality depends critically on retrieval: weak embeddings surface irrelevant documents, and the answer degrades with them.

Web-scale RAG adds two layers. The index is a search engine rather than a private vector store, and the query sent to it is rarely the one the user typed. Assistants rewrite a prompt into several sub-queries, a technique Google calls query fan-out. Peec AI's analysis of 5 million fan-out queries collected in April 2026 put ChatGPT at about 2.1 searches per prompt against roughly nine for Gemini 3, with ChatGPT merging result lists through Reciprocal Rank Fusion. When a fresh page is needed, a user-triggered fetcher retrieves it. Anthropic's Claude-User is one example, which the company separated from its training crawler in documentation published on February 25, 2026.

Model providers, AI search products and enterprises run retrieval; publishers and brands own the pages retrieved; managed services sit between them. Cloudflare opened AutoRAG in beta on April 7, 2025. "Building a RAG pipeline is a patchwork of moving parts," Anni Wang of Cloudflare said at the launch.

Origin and evolution

Retrieval-backed language models predate the acronym. Google researchers led by Kelvin Guu posted REALM on February 10, 2020, pre-training a model jointly with a retriever over Wikipedia; according to the paper, it beat previous methods on three open-domain question-answering benchmarks by 4 to 16 points of absolute accuracy. The name arrived three months later. Patrick Lewis and eleven co-authors from Facebook AI Research, University College London and New York University posted "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks" on May 22, 2020. Their system paired a pre-trained sequence-to-sequence generator with a dense vector index of Wikipedia. The abstract named two problems it addressed: updating a model's knowledge and showing provenance. The paper was presented at NeurIPS 2020, and its final arXiv revision is dated April 12, 2021.

ChatGPT's release in November 2022 created demand for chatbots that answer from company documents, and constraints surfaced quickly. Stanford-led researchers reported in July 2023 in "Lost in the Middle" that models used information best at the start or end of a prompt and degraded when it was buried in between. Chunking created a second problem: a passage stating that revenue grew 3% loses its meaning once the company name has been split into a different chunk.

Anthropic's answer, published on September 19, 2024 as contextual retrieval, prepended a short machine-written summary to each chunk before indexing. The company reported that the change cut failed retrievals by 49%, and by 67% when combined with reranking, taking the top-20 failure rate from 5.7% to 1.9%. Those are vendor figures measured on the company's own test sets. The same post set out a threshold: knowledge bases smaller than 200,000 tokens, roughly 500 pages, can be placed in the prompt whole, with no retrieval at all.

Why it matters for marketers

RAG changed what a search index is for. Google's own AI search documentation, reviewed by consultant Marie Haynes in May 2026, describes RAG as the process by which AI features use websites to ground answers. Without crawlable, indexed content, the model has nothing to cite. That makes retrieval, rather than the model, the gatekeeper of AI visibility. When ChatGPT referrals fell 52% in mid-2025, Reddit citations rose 87% and Wikipedia's 62% within a month, a shift analysts attributed to OpenAI reweighting its RAG system.

An optimisation trade has grown around the retrieval step, and Google has pushed back on parts of it. Guidance published in July 2026 and summarised in PPC Land's AI Mode explainer states that llms.txt files, content chunking and rewriting text for AI systems do nothing for visibility in Search.

Inside advertising, RAG is becoming plumbing. IAB Tech Lab's AI in advertising primer, released in October 2024, presented it as a way to produce more accurate, contextually relevant content. The Ad Context Protocol (AdCP), launched on October 15, 2025, allows around 60 seconds for product discovery because that task involves inference and retrieval, against about one second for a format listing served from a database. A paper covered in April 2026 argued that RAG systems which pre-tag documents with taxonomy labels outperform pure semantic search on precision.

The commercial conflict turns on payment. Training reads a page once; retrieval can read it every time an answer is composed. Cloudflare's crawler dashboard classifies bots as Training, Search or Agent, with Search covering bots that refresh RAG databases. IAB Tech Lab's Content Monetization Protocols (CoMP) working group, formed in August 2025 under the name LLM Content Ingest API, released CoMP v1.0 for public comment on March 10, 2026, requiring commercial agreements before content is crawled or used.

Limitations and disputes

Retrieval narrows hallucination; it does not end it. A preregistered study by Stanford RegLab of legal research tools marketed on their RAG architecture found that each hallucinated between 17% and 33% of the time, according to the paper. Lexis+ AI answered 65% of queries accurately and Westlaw AI-Assisted Research 42%. Both vendors disputed the methodology.

Answers inherit the quality of the corpus. SEO consultant Lily Ray argued in June 2026 that RAG systems treat repetition as a proxy for consensus, allowing misinformation repeated across enough pages to become the answer.

Copyright is the largest open dispute. The US Copyright Office's May 2025 report noted that RAG involves copying works into retrieval databases and flagged substitution where a retrieved answer satisfies demand for the original. Encyclopaedia Britannica and Merriam-Webster sued Perplexity on September 10, 2025, alleging that articles were copied as RAG inputs; CNN's complaint followed in May 2026, covering more than 17,000 works.

Regulators have begun to separate retrieval from training. The UK Competition and Markets Authority's Publisher Conduct Requirement, imposed on June 3, 2026, obliges Google to offer opt-outs covering grounding as well as training, with page-level controls due on March 3, 2027.

Not the same as

Grounding is the commercial term for the same idea. PPC Land's grounding explainer treats RAG as the architecture and grounding as the product category built on it, with vendors reserving the latter for managed services such as web search.

Fine-tuning alters a model's weights through additional training. RAG leaves the weights untouched, so a fact can be corrected by re-indexing a document rather than retraining anything.

Training data collection gathers text to build a future model. Retrieval fetches text to answer a present question, which is why operators run separate crawlers and why blocking one does not block the other.

Long-context prompting places an entire document set inside the prompt. Below roughly 200,000 tokens, Anthropic presents it as an alternative to retrieval; above that, selection becomes unavoidable.

Recent developments

TollBit measurement published in August 2026 found that 15% of AI page fetchers in Europe reached disallowed URLs, with ChatGPT-User reaching blocked pages on nearly half the European sites that named it in robots.txt. Cloudflare's revised defaults, which block Training and Agent crawlers on ad-carrying pages for newly onboarded domains, took effect on September 15, 2026.

On the reliability side, a study posted to arXiv the same day by researchers from the University of Bonn, Fraunhofer IAIS, the Lamarr Institute and Microsoft Germany found that replacing personal data with placeholder tags before prompting dragged GPT-4o mini's score on a RAG benchmark from 0.80 to 0.32, while reasoning tasks barely moved. Retrieval, in other words, is where the privacy trade-off bites hardest.

Timeline

  • February 10, 2020: Google researchers post REALM, a language model pre-trained alongside a Wikipedia retriever
  • May 22, 2020: Patrick Lewis and eleven co-authors post the paper that names retrieval-augmented generation
  • December 2020: The RAG paper is presented at NeurIPS 2020
  • April 12, 2021: Final arXiv revision of the RAG paper
  • November 2022: ChatGPT's release drives demand for document-grounded chatbots
  • July 6, 2023: Stanford-led researchers post "Lost in the Middle" on position effects in long prompts
  • May 2024: Stanford RegLab preprint finds RAG-based legal tools hallucinate 17% to 33% of the time
  • June 2024: Grounding with Google Search reaches general availability on Vertex AI
  • September 19, 2024: Anthropic publishes contextual retrieval
  • October 2024: IAB Tech Lab publishes its AI in advertising primer covering RAG
  • April 7, 2025: Cloudflare opens AutoRAG in beta
  • May 2025: US Copyright Office report addresses copying in retrieval databases
  • August 2025: IAB Tech Lab forms the working group later named CoMP
  • September 10, 2025: Britannica and Merriam-Webster sue Perplexity
  • October 15, 2025: Ad Context Protocol launches
  • February 25, 2026: Anthropic documents the roles of ClaudeBot, Claude-User and Claude-SearchBot
  • March 10, 2026: IAB Tech Lab releases CoMP v1.0 for comment
  • May 28, 2026: CNN sues Perplexity over more than 17,000 works
  • June 3, 2026: CMA imposes its Publisher Conduct Requirement on Google
  • July 2026: Google states that chunking and llms.txt do not affect Search visibility
  • September 15, 2026: Anonymisation study posted to arXiv; Cloudflare's new crawler defaults take effect
  • December 3, 2026: Main obligations of the CMA requirement take legal force
  • March 3, 2027: Page-level grounding controls due under the CMA requirement

Summary

Who: Model providers, AI search products and enterprises run retrieval systems; publishers, retailers and brands own the content retrieved; infrastructure firms such as Cloudflare and standards bodies such as IAB Tech Lab sit between them. The term was coined by Patrick Lewis and eleven co-authors at Facebook AI Research, University College London and New York University.

What: An architecture in which a language model receives passages fetched from an external index at query time and writes its answer from them. Documents are chunked, embedded and stored; queries are embedded, matched, reranked and inserted into the prompt; the model's weights stay unchanged.

When: Named in a paper posted on May 22, 2020, following Google's REALM in February 2020. Commercialised from 2023 and 2024, and litigated and regulated from 2025, with UK page-level grounding controls due on March 3, 2027.

Where: Inside AI Overviews, AI Mode, ChatGPT search, Perplexity, enterprise assistants and advertising agents, with retrieval reaching across the open web and private document stores.

Why: Trained models are static, cannot cite and invent answers. Retrieval supplies freshness and provenance, and in doing so decides which pages are read, cited and credited, a question now contested in courtrooms, standards bodies and competition authorities.