Security firm GreyNoise reported on August 28, 2026 that automated scanners spread across 824 internet addresses forged the names of 13 AI crawlers belonging to eight companies, requesting environment files, cloud access keys, private keys and password stores from misconfigured web servers.
The finding lands on a control that most publishers, advertisers and platform operators already rely on without examining it closely. A crawler declares itself in the user agent line of an HTTP request. Anthropic publishes the string ClaudeBot. OpenAI publishes GPTBot and its siblings. Site owners write those names into robots.txt files, firewall rules and analytics filters. Nothing in the request itself proves the name is accurate.
According to GreyNoise, a cluster of scanners exploited exactly that gap. Six of the forged crawler names arrived from the same 824 addresses in almost identical volume, and within that cluster not one of the six requested /robots.txt at any point.
What the traffic looked like
The measurement window ran from July 28 to August 23, 2026. Across it, six AI crawler names belonging to four companies arrived on a single HTTP client fingerprint, according to the company. A fingerprint of that type identifies the software making the request rather than the machine running it, which is why traffic from 824 separate addresses could be grouped as one population.
That same fingerprint carried more than 1,500 different user agent strings across the 90 days to August 23, most of them ordinary browsers. The six crawler names were a subset of a much larger rotation. Almost all of the six-name traffic arrived in August, with the largest single day recorded on August 23, the final day of the window.
The four companies whose crawler names appeared in the matched cluster are Anthropic, OpenAI, Google and Perplexity. GreyNoise stated that the impostor ClaudeBot string matched Anthropic's own character for character, meaning no rule written against the user agent could separate the two.
Volume alone was not the tell. The behaviour was.
The robots.txt signal
A crawler that intends to read pages fetches /robots.txt first, because that file states what the site permits. Under the six forged names, according to GreyNoise, that file was never requested once.
The comparison the company drew is the useful part. Anthropic's real crawler, measured over the same window by the same method, requested /robots.txt more than any other single path, at 12 percent of its traffic, and never requested a credential file. The impostors inverted both figures.
What the forged traffic asked for instead was credentials. Across all traffic on the fingerprint, which carried names beyond the six, requests for environment files, cloud access keys, private keys and password stores ran into the millions. An environment file is where an application stores database passwords, cloud access keys and API tokens, which makes it a single-request route to an organisation's infrastructure.
The most requested impostor paths listed in the report were /.env, /app/.env, /api/.env, /backend/.env, /.env.local, /.env.production, /.env.old, /.env.bak, /.aws/credentials and /.env.swp. GreyNoise associated the activity with existing tags for environment file crawling, generic sensitive file access attempts, AWS configuration scanning, Git config crawling, EC2 IAM credential access attempts, and attempts against CVE-2025-30208, an arbitrary file disclosure issue in the Vite build tool. The post lists Vite versions 6.2.3, 6.1.2, 6.0.12, 5.4.15 and 4.5.10 as the patched releases.
A token that has no user agent
One number in the report stands apart from the rest. Google-Extended, the robots.txt token publishers write to opt out of AI training, generated 263,849 sessions in the observed traffic. Every one of them was forged by definition, because Google-Extended is not a crawler at all. Google's own documentation, quoted in the report, states that it "doesn't have a separate HTTP request user agent string." No Google crawler sends it.
The distinction between a control token and a crawler has been documented before. IAB Australia drew the same line in guidance published in July 2026, separating directives such as Google-Extended and Applebot-Extended, which never independently visit a site, from the crawlers that actually make requests. A scanner sending Google-Extended as a user agent was therefore announcing a string that has no legitimate sender anywhere on the internet, and it still produced more than a quarter of a million sessions.
Google added Google-Extended to robots.txt on September 28, 2023 as a training opt-out. Its documentation confirms that restricting it does not affect a site's inclusion in Search and that it is not used as a ranking signal.
How the check was made
Address verification is the control the report tests. All four companies publish the ranges their crawlers operate from, in machine-readable JSON files. GreyNoise fetched every one of those lists, along with Amazon's, and checked every address that sent a forged name against all of them.
Not one matched.
The lists cited in the post cover Anthropic's crawler bot file, OpenAI's separate files for GPTBot, ChatGPT-User and OAI-SearchBot, Google's three crawler range files for common crawlers, special crawlers and user-triggered fetchers, Perplexity's two files, and Amazon's two address lists for Amazonbot and its search bot. Over the same window, thousands of sessions carrying the ClaudeBot name did arrive from Anthropic's published range at 216.73.216.0/22.
Two further details limit what defenders can do with network controls. The 824 addresses sit inside 795 separate /24 networks, which leaves no single block to filter. And almost every impostor session carried the same web crawler classification label that genuine crawler traffic carries, so category-based filtering does not separate them either.
GreyNoise also published the impostor client fingerprint in half-redacted form, ge11nn05enus_f3bb7a, alongside the genuine Anthropic fingerprint ge11nn080000_757a95, and recommended the former for investigation rather than blocking. The company noted that neither signal is sufficient alone: the real crawler and the impostor both send the identical ClaudeBot user agent string, so access is warranted only when fingerprint and published address agree.
Amazon's names, in greater volume
Beyond the four companies in the matched cluster, the scanners forged two of Amazon's crawler names in even greater volume, according to GreyNoise. Neither of those strings matches the user agent Amazon documents, which places them in the same category as Google-Extended: names that no legitimate Amazon crawler sends.
That detail sits alongside a live dispute over Amazon's own crawler behaviour. PatronView blocked Amazonbot in August 2026 after recording 117,000 daily page reads, and Amazon is separately litigating against Perplexity in the Ninth Circuit over agents that allegedly impersonated human users through user agent spoofing, a case in which Digital Content Next filed an amicus brief on April 29, 2026 on behalf of publishers reaching 259 million unique visitors.
An inconsistency in the framing
The report's headline names OpenAI, Anthropic and DeepSeek as the impersonated firms. The body of the report does not name DeepSeek among the four companies whose crawler names formed the matched six-name cluster, which are given as Anthropic, OpenAI, Google and Perplexity. The wider cluster is described as covering 13 AI crawlers from eight companies, but the eight are not individually enumerated in the published post, and DeepSeek appears only in the title and in the summary distributed to press.
The report is equally explicit about what it does not establish. GreyNoise observes requests arriving. Nothing in the findings indicates that a requested file was returned, that any organisation was compromised, or who operates the scanners. The company stated that it is not attributing the activity.
Why this reaches marketing teams
The immediate exposure is not limited to security operations. Three separate marketing functions currently treat crawler names as trustworthy input.
The first is access policy. Publishers spent 2025 and 2026 building differentiated robots.txt files that permit some AI crawlers and refuse others, a practice with measurable revenue consequences on both sides. Research from Rutgers Business School and The Wharton School found news publishers who blocked model crawlers lost roughly 7 percent of weekly traffic within six weeks, with a larger figure of 23.1 percent of monthly visits reported in an earlier version of the same work. Any allowlist that grants access on the strength of a name grants that access to an impostor sending the same name.
The second is measurement. Bot classification feeds traffic reporting, and traffic reporting feeds media planning. Cloudflare Radar telemetry placed automated systems at 57.4 percent of HTTP requests for web content on June 3, 2026, against 42.6 percent from people. Decodo analysis published in August 2026 put the United States at 53.5 percent of global bot traffic, with retail absorbing 13 percent of automated requests. Traffic that carries a recognised crawler label but is neither a crawler nor a person sits in the gap between those categories.
The third is spend protection. Invalid traffic remains a measured drain on performance budgets even as verification vendors report improvement. DoubleVerify reported on July 29, 2026 that fraud and sophisticated invalid traffic violation rates fell 41 percent year over year in North America and 45 percent in Europe, the Middle East and Africa among protected campaigns, while finding that AI bots generated up to ten times more clicks than humans in some unprotected campaigns. A Lunio survey of 131 senior marketers published on July 15, 2026 found that 75.6 percent estimate losing more than 5 percent of monthly performance budget to bots while only 5.3 percent run a dedicated invalid traffic platform.
The verification gap was already documented
None of the underlying weakness is new, which is what makes the volume notable. The user agent has been described as unreliable for years, and the industry response has been partial.
Google moved its crawler IP range files from a weekly to a daily refresh schedule on March 18, 2025, explicitly to narrow the window in which a stale allowlist can be exploited. Gary Illyes, an analyst at Google, said at the time that the change followed feedback from large network operators, and the documentation frames the files as a defence against parties claiming to be Googlebot. That refresh cadence is the benchmark other operators are measured against. Apple added 4,656 addresses to its Applebot prefix file in a single August 2026 update without a changelog or an accurate internal timestamp, which left allowlists frozen at the previous state missing two thirds of the advertised pool.
Cryptographic verification exists as a proposal rather than a deployed standard. Cloudflare shared its Web Bot Auth scheme in May 2025, using HTTP Message Signatures and public key cryptography, and published a registry format for discovering and validating signing agents on October 30, 2025. Google referenced an experimental web-bot-auth mechanism when it added Google-Agent to its crawler documentation on March 20, 2026. Adoption across AI operators remains limited, which leaves published address lists as the practical verification method for most sites.
Spoofing has been documented in operational traffic throughout the period. Research covered in January 2026 recorded AI agents presenting spoofed user agent strings to bypass site defences, with a single query to one chatbot triggering 16 requests from 12 addresses impersonating human browsers. DataDome tested 698,214 live websites with a spoofed assistant user agent and found that 79.7 percent did not block or challenge the impersonation, with 79.2 percent of those returning a 200 OK response. Cloudflare accused Perplexity of stealth crawling through undeclared crawlers disguised as a standard browser in August 2025, counting 3 to 6 million daily undeclared requests alongside 20 to 25 million declared ones.
What separates the GreyNoise findings from that record is intent. The earlier cases concerned commercial actors taking content they had been refused. This traffic was not reading pages at all. It was enumerating credential paths behind a name that publishers had been told to trust.
What the report recommends
GreyNoise directs defenders to verify identity through more than the declared name, checking the connecting address against the published list for the name a request claims wherever a crawler name already grants access or waives a control. For security operations, the post flags alerting on requests for /.env, /.aws/credentials and /.git/config, and treating a crawler that never requests /robots.txt across multiple days as suspect, since real crawlers cache that file rather than fetching it every visit.
For platform administrators, the recommendations cover keeping environment files, Git directories and cloud credential files outside the web root, rotating any cloud key that was ever reachable from a web path, and updating Vite to a patched release. For security leadership, the post calls for an owner and a refetch schedule for each vendor address list, on the reasoning that a stale list converts a legitimate crawler into a false alert.
The full list of 824 addresses is published with the report, with the complete indicator set, credential paths and JA4+ fingerprints distributed to GreyNoise customers.
Timeline
- September 28, 2023 - Google adds Google-Extended as a robots.txt token for AI training and grounding, with no associated user agent string
- March 18, 2025 - Google switches its crawler IP range files from a weekly to a daily refresh schedule following feedback from large network operators
- May 2025 - Cloudflare shares the Web Bot Auth proposal, introducing cryptographic authentication for automated traffic
- August 2025 - Cloudflare accuses Perplexity of stealth crawling using undeclared crawlers disguised as a standard browser
- October 30, 2025 - Cloudflare publishes a registry format for discovering and validating bots and agents that sign their requests
- January 6, 2026 - Research documents AI agents using spoofed user agents and distributed address rotation to bypass site defences
- February 25, 2026 - Anthropic separates the functions of ClaudeBot, Claude-User and Claude-SearchBot in updated documentation
- March 7, 2026 - DataDome testing of 698,214 websites finds 79.7 percent do not challenge a spoofed AI assistant user agent
- March 20, 2026 - Google adds Google-Agent to its crawler documentation with a reference to an experimental web-bot-auth protocol
- April 26, 2026 - Updated Rutgers and Wharton research puts the cost of blocking AI crawlers at roughly 7 percent of weekly traffic within six weeks
- July 22, 2026 - IAB Australia publishes crawler guidance separating control tokens such as Google-Extended from crawlers that make requests
- July 28, 2026 - Start of the GreyNoise measurement window for the six forged AI crawler names
- July 29, 2026 - DoubleVerify reports fraud and invalid traffic violation rates down 41 percent in North America and 45 percent in EMEA among protected campaigns
- August 11, 2026 - Decodo analysis places the United States at 53.5 percent of global bot traffic
- August 23, 2026 - Largest single day of forged crawler traffic recorded, and final day of the measurement window
- August 28, 2026 - GreyNoise publishes its findings on scanners forging 13 AI crawler names from eight companies
Related PPC Land coverage
- Anthropic clarifies what its three web crawlers do - and how to block them - Documentation update separating ClaudeBot, Claude-User and Claude-SearchBot and setting out what blocking each one costs a site.
- The user agent strings every SEO and site owner needs right now - Reference list of published crawler tokens with the standing caveat that string matching alone cannot establish identity.
- AI agents caught masquerading as humans to bypass website defenses - Documented spoofing of browser user agents and address rotation by commercial AI agents.
- Google updates crawler verification processes with daily IP range refreshes - The move to daily JSON refreshes and the reverse DNS verification method behind it.
- Apple adds 4,656 IP addresses to Applebot crawler in one update - How a silent expansion of a published prefix file breaks allowlists in both directions.
- Cloudflare unveils registry format for bot and agent authentication - The registry layer built on Web Bot Auth signatures for discovering and validating agent identities.
- Google-Agent joins the crawler list as AI browsing gets an official identity - Google's crawler documentation addition and its reference to cryptographic bot authentication.
- IAB Australia forces every crawler into one of four verdicts - Trade body guidance on classifying crawlers and the instability of any token-based allowlist.
- AI bots crawl retail sites 198x more than Google, new report warns - Botify and DataDome research including the 698,214-site spoofing test.
- Perplexity denies training AI models as Cloudflare documents stealth crawlers - Declared and undeclared crawler volumes at the centre of the 2025 identity dispute.
- US sends 53.5% of global bot traffic, Decodo analysis finds - Distribution of automated traffic by origin and by sector.
- DoubleVerify: ad fraud drops 41% in North America, 45% in EMEA - Verification data on invalid traffic rates and AI bot click volumes in unprotected campaigns.
- Only 5.3% of marketers use IVT tools as 75.6% lose ad budget to bots, Lunio - Survey evidence on the gap between concern about bot traffic and investment in detecting it.
- Blocking AI crawlers cost news publishers 7% of traffic, study finds - Rutgers and Wharton measurement of what refusing crawler access costs in human traffic.
- PatronView blocks Amazon's AI crawler after 117,000 daily page reads - A publisher's account of crawl volume, firewall blocking and crawler response.
- Why major publishers are backing Amazon against Perplexity's AI spoofing - The Ninth Circuit dispute over agents impersonating human users through user agent strings.
Summary
Who: GreyNoise, a security intelligence company that operates internet-wide sensor infrastructure, published the research. The crawler names forged in the observed traffic belong to Anthropic, OpenAI, Google, Perplexity and Amazon, with the wider cluster covering 13 AI crawler names from eight companies. The parties affected are website operators, publishers, advertisers and platform administrators who grant access or waive controls on the basis of a declared crawler name.
What: A cluster of automated scanners distributed across 824 internet addresses in 795 separate /24 networks forged AI crawler user agent strings while requesting environment files, cloud access keys, private keys and password stores. Six of the names arrived on a single HTTP client fingerprint in matched volume, none requested /robots.txt, and no address matched any of the crawler IP range files the four companies publish. Google-Extended, a robots.txt token that has no associated user agent string, accounted for 263,849 forged sessions.
When: The measurement window ran from July 28 to August 23, 2026, with almost all six-name traffic recorded in August and the peak on August 23. GreyNoise published the findings on August 28, 2026. The fingerprint carried more than 1,500 distinct user agent strings across the 90 days to August 23.
Where: The activity was observed against web servers globally through GreyNoise sensor traffic. The 824 source addresses spread across 795 separate /24 networks, and none fell inside the published ranges maintained by Anthropic, OpenAI, Google, Perplexity or Amazon.
Why: The user agent is a client-supplied header, so any control that checks a crawler name without checking the connecting address can be bypassed by copying the name. Published IP range files remain the only verification method in wide deployment, since cryptographic alternatives such as Web Bot Auth have limited adoption among AI operators. The finding matters commercially because crawler names now govern content licensing decisions, traffic measurement and invalid traffic filtering, all of which inherit the weakness.
Discussion