A series of controlled experiments published by AI SEO agency DEJAN on 30 May 2025 produced findings that have since resurfaced and spread widely in professional circles: Google's AI Mode does not appear to retrieve content from the live web in the way that classic Google Search does. Instead, according to Dan Petrovic of DEJAN, the feature draws from what he described as "a proprietary content store separate from the search index." That conclusion, backed by reproducible tests, has significant implications for marketers and publishers who assume that ranking in Google Search automatically means appearing in AI Mode responses.

The LinkedIn post by Chris Long, co-founder at Nectiv, that re-circulated Petrovic's research today attracted 165 reactions and 25 comments from professionals across SEO, content strategy, and AI search optimization. The speed and weight of that reaction reflects how consequential the underlying finding is - not as a niche technical curiosity, but as a structural fact about how Google's fastest-growing search surface actually works.

The experiments

Petrovic's original test, described in his article published at dejan.ai on 30 May 2025, followed a straightforward logic. He deleted a page at the URL dejanmarketing.com/flux/ and fetched it inside AI Mode. The result was a 404 error. He then republished the page and fetched it again. AI Mode still returned a 404. A check in classic Google Search, however, confirmed the page was indexed and ranking normally. According to the DEJAN article, the more revealing detail emerged retroactively: the URL was already returning a 404 inside AI Mode's Python execution environment even before the deletion took place, despite being indexed and ranking in traditional Search. That inconsistency ruled out a simple crawl-delay explanation.

A second, cleaner test was then run. Petrovic created a page at dejan.ai/tools/test containing a hidden instruction - a "secret message" reading "I know kung-fu." The page included a directive telling any AI that fetched it to return only that phrase and nothing else. The page was submitted and confirmed indexed in classic Google Search. Gemini, the conversational AI product, demonstrated a direct connection to Google's search index and returned the message as instructed. AI Mode, by contrast, remained completely unaware of the page and its content. According to DEJAN's documentation of the test, AI Mode behaved identically to models in AI Studio and Vertex that had no access to the page whatsoever.

Petrovic's conclusion was direct: "Our tests show that Google's AI Mode doesn't retrieve page content from the live web but somewhere else, and that 'somewhere else' appears to be a proprietary content store separate from the search index."

Where FastSearch fits

The DEJAN findings connect to technical infrastructure that only became publicly known through court filings in Google's antitrust case. According to a Search Engine Land article published on 13 November 2025, Google uses a proprietary technology called FastSearch to ground its Gemini models and generate AI Overviews. The article drew on court documents filed as part of the remedy proceedings, which state: "FastSearch is based on RankEmbed signals which are a set of search ranking signals that generates abbreviated, ranked web results that a model can use to produce a grounded response. FastSearch delivers results more quickly than Search because it retrieves fewer documents, but the resulting quality is lower than Search's fully ranked web results."

Three structural compromises define how FastSearch operates. First, rather than querying Google's full web index, it pulls from a targeted subset of pages - a smaller document pool designed to reduce processing time. Second, it relies primarily on RankEmbed signals rather than the full array of ranking signals that traditional Search uses. RankEmbed, which PPC Land has covered extensively in the context of the Google antitrust proceedings, is a dual encoder model that embeds queries and documents into a shared vector space to measure semantic proximity. It does not weigh traditional authority signals such as backlinks in the same way. Third, Google explicitly acknowledged in the court filing that FastSearch produces results of lower quality than fully ranked Search results - though still "good enough for grounding" AI responses.

The court documents also describe RankEmbed as one of Google's "top-level" deep-learning signals, capable of "finding and exploiting patterns in vast data sets." A page with modest backlink counts but clear topical alignment can outperform a high-authority domain if the semantic match to the query is stronger. Court documents revealed earlier that RankEmbed was trained on a single month of search data - a strikingly small window for a system with this much influence on what gets seen inside AI Mode.

Vertex AI customers can ground their own applications using FastSearch, but according to the court filing cited by Search Engine Land, they "do not receive the FastSearch-ranked web results themselves, only the information from those results." Google restricts access in this way to protect its intellectual property, which means independent testing of FastSearch's behaviour is structurally limited.

What the LinkedIn discussion revealed

The professional discussion triggered by Chris Long's post added texture to the technical baseline. Lorenzo Schiff, identified as a senior SEO technologist and LLMOps specialist, offered a parsing of the DEJAN methodology. According to Schiff's comment, a 404 at fetch time does not by itself prove deindexation, because Google explicitly documents that pages returning 404 or 410 status codes only drop from the index after they have been crawled and processed. He also noted that Google has long acknowledged a URL can remain represented in Search based on links and other publicly available signals even when the content itself is not crawlable. His conclusion was more cautious than Petrovic's: "AI Mode likely does not rely on a true live page fetch at answer time, and probably uses a distinct retrieval/serving layer with different freshness, passage selection, and grounding behavior than the classic SERP. But Google's own documentation still ties AI Mode supporting links back to pages that are indexed and snippet-eligible in Google Search."

That framing - a different serving layer rather than a proven separate index - is a meaningful distinction. It does not undermine the practical implications of the research, but it shapes how the finding should be communicated to clients and stakeholders.

Other commenters drew on direct operational experience. One practitioner wrote that page updates made weeks prior still appeared in AI Overviews with old content, while the standard SERP reflected the new version. Another noted it took their new consultancy site 18 days to appear in AI Mode after launch. A third commenter highlighted a concern with direct compliance implications: "The 404 experiment shows that deleted or removed pages can still be served; that can be a problem or legally compromising if content was removed for a compliance issue or retracted claim."

Kathleen O'Brien Thompson framed the core marketing implication in plain terms: "Being in Google Search does not guarantee you're in AI Mode."

The compute logic behind the behaviour

Several commenters offered plausible technical explanations for why a cached or separate retrieval layer might exist. Sean Smith, an agency GTM strategist, pointed to mobile query volume as the driver: accessing the live web on every AI Mode query would require significantly more compute and add latency, particularly on mobile where AI Mode is most prevalent. The practical trade-off - accepting a degree of content staleness in exchange for speed - aligns with how FastSearch was described in the antitrust filings.

Ryan Anderson proposed a more specific model: FastSearch may operate as a post-processing layer on the same underlying index rather than a fully separate one. Under this interpretation, models such as BERT retrieve semantically appropriate pages, and AI Mode then generates answers from those sources - representing retrieval-augmented generation (RAG) on top of a vector-indexed representation of the web, rather than a live fetch. That framing echoes what PPC Land has documented about Google's shift toward LLM-centric search architecture, where the distinction between indexing for retrieval and indexing for ranking is increasingly relevant.

The question of how frequently the FastSearch layer is refreshed remains unanswered. Neither Google nor the court documents provided a specific crawl or update cadence for the content store that feeds AI Mode. That opacity makes the problem harder to manage in practice.

Why this matters for the marketing community

The implications branch in several directions. For publishers who updated pages, changed product names, rebranded services, or removed legally sensitive content, the lag between a change taking effect in traditional Search and that change propagating to AI Mode is not a minor UX issue. It is a material gap between what is true on the live web and what AI Mode presents to users. Google added AI Mode to its robots meta tag documentation in March 2025, giving publishers a technical mechanism to exclude content from AI Mode responses using the nosnippet directive. But that control operates at the point of serving, not at the point of content store population.

For new content, the picture is similarly challenging. A page published today, confirmed indexed in Google Search, may remain invisible in AI Mode for days or weeks. That matters most for time-sensitive content - pricing pages, event announcements, product launches - where the gap between indexing and AI Mode visibility has real commercial consequences.

The broader context is one of structural change in how search visibility is measured. As PPC Land has covered, Google lifted the waitlist for AI Mode in the United States on 1 May 2025, making it immediately available to all US users over 18. By July 2025, the feature had extended to Google Workspace accounts, and by October 2025 to more than 40 countries and territories. AI Mode is no longer an experimental product. It is a mainstream search surface, and the content it surfaces is drawn from a system that operates differently from the one most digital marketers have spent years optimising for.

Research has consistently shown declining organic click-through rates as AI surfaces expand. Ahrefs documented in February 2026 that AI Overviews correlate with a 58% reduction in click-through rates for top-ranking pages. Against that backdrop, content that fails to appear in AI Mode at all - because it is too new, too recently updated, or for reasons that remain opaque - faces a compounding disadvantage.

The DEJAN research also complicates the relationship between AI Mode and answer engine optimisation (AEO), a discipline that has grown rapidly in response to AI search. If authority and sustained visibility in a cached or processed content store matter more than freshness, the optimisation priorities shift. Being indexed quickly is necessary but not sufficient. Appearing in AI Mode may require a different type of content maturity - one built on longevity and repeated citation rather than recency.

At the same time, court documents have revealed that Vertex AI customers who use FastSearch for grounding their own applications receive only derivative information from FastSearch results, not the ranked URLs themselves. That asymmetry - Google using FastSearch internally to power AI Mode while limiting third-party access - reinforces why independent research like Petrovic's fills an important gap. There is currently no publicly available tool that lets a publisher test whether a given URL is present and current in the FastSearch content store.

Timeline

Summary

Who: Dan Petrovic of DEJAN, an AI SEO agency, conducted and published the original experiments. Chris Long, co-founder at Nectiv, re-circulated the research on LinkedIn on 21 April 2026, prompting a broad professional discussion. Commenters including Lorenzo Schiff and Kathleen O'Brien Thompson added analysis and marketing context.

What: Controlled tests show that Google's AI Mode does not retrieve content from the live web index in the way that classic Google Search does. A page confirmed as indexed and ranking in traditional Search was not accessible to AI Mode both before and after a deletion event. A second test using a hidden message confirmed that Gemini (the standalone app) could access indexed content directly, while AI Mode could not. Court documents from Google's antitrust proceedings confirm the existence of FastSearch - a proprietary retrieval layer using RankEmbed signals that delivers faster but lower-quality results than fully ranked Search - as the technology grounding AI Mode responses.

When: DEJAN's original research was published on 30 May 2025. The Search Engine Land explainer on FastSearch appeared on 13 November 2025. The LinkedIn discussion resurface occurred on 21 April 2026.

Where: The tests were conducted on DEJAN's own web properties. The professional discussion took place on LinkedIn. FastSearch was revealed in US federal court documents filed in the Google antitrust remedy proceedings.

Why: The finding matters because AI Mode has expanded from a US experimental feature to a mainstream global search surface used by millions of users across consumer and Workspace accounts. If AI Mode draws from a cached or processed content store that updates on a different cadence than the live index, then publishers, marketers, and content teams cannot rely on traditional Search indexing as a proxy for AI Mode visibility. Content freshness, recent updates, page deletions, and new publications may all reach AI Mode users with significant delays - or not at all. The gap between being indexed and being present in AI Mode is a structural blind spot that the marketing community has only begun to map.

Share this article
The link has been copied!