By Luís Daniel in Search — Sep 7, 2020 Microsoft launches a new robots.txt tester tool for Bing robots.txt is a file on the domain/subdomain that informs robots/crawlers/spiders which sections of a website should not be processed or scanned. robots.txt tester