By Luís Daniel Rijo in Search News — Sep 7, 2020

Microsoft launches a new robots.txt tester tool for Bing

robots.txt is a file on the domain/subdomain that informs robots/crawlers/spiders which sections of a website should not be processed or scanned.

robots.txt tester

Microsoft last week launched a new tester tool for robots.txt. The robots.txt tester tool enables webmasters to analyse their robots.txt file and it highlights the issues that would prevent utls to be crawled by Bing and other search index robots.

robots.txt is a file on the domain/subdomain that informs robots/crawlers/spiders which sections of a website should not be processed or scanned.

Microsoft says that the the growing number of search engines and parameters have forced webmasters to search for their robots.txt file amongst the millions of folders on their hosting servers, editing them without guidance and finally scratching heads as the issue of that unwanted crawler still persists.

With the new tester tool for robots.txt, webmasters can submit a URL and Microsoft checks the robots.txt file and verifies if the URL has been allowed or blocked.

PPC Land is an international news publication headquartered in Frankfurt, Germany. PPC Land delivers daily articles brimming with the latest news for marketing professionals of all experience levels.

Subscribe to our newsletter for just $10/year and get marketing news delivered straight to your inbox. By subscribing, you are supporting PPC Land.

You can also follow PPC Land on LinkedIn, Bluesky, Reddit, Mastodon, X, Facebook, Threads, and Google News.

Know more about us or contact us via info@ppc.land

Our latest marketing news:

Subscribe via email