By Luís Daniel Rijo in Search News — Nov 27, 2023

Google Search Console crawl rate limiter tool to be deprecated

Google last week announced the Crawl Rate Limiter Tool in Google Search Console will be deprecated on January 8, 2024.

Crawler

Google last week announced the Crawl Rate Limiter Tool in Google Search Console will be deprecated on January 8, 2024.

The Crawl Rate Limiter Tool in Google Search Console was a feature that allowed site owners to control how often Googlebot crawls their website.

This tool has been available for over a decade, but with advancements in crawling logic and other tools, its usefulness has diminished.

Googlebot, Google's web crawling bot, reacts to how a website's server handles its HTTP requests. If the server consistently returns HTTP 500 status codes or if response times get significantly longer, Googlebot will automatically slow down crawling.

The Crawl Rate Limiter Tool had a much slower effect on crawling speed, and it was rarely used by site owners. With the deprecation of this tool, Google is setting the minimum crawling speed to a lower rate, ensuring that crawlers don't waste a site's bandwidth.

Google is also keeping the Googlebot report form for reporting unusual Googlebot activities and for emergency cases. The fastest way to reduce crawl rate is to instruct Googlebot through server responses.

PPC Land is an international news publication headquartered in Frankfurt, Germany. PPC Land delivers daily articles brimming with the latest news for marketing professionals of all experience levels.

Subscribe to our newsletter for just $10/year and get marketing news delivered straight to your inbox. By subscribing, you are supporting PPC Land.

You can also follow PPC Land on LinkedIn, Bluesky, Reddit, Mastodon, X, Facebook, Threads, and Google News.

Know more about us or contact us via info@ppc.land

Our latest marketing news:

Subscribe via email