Back in May, we launched OpenRobotsTXT, a project to archive and analyse the world’s robots.txt The post New: Site Search and Bot Lookup for OpenRobotsTXT appeared first on Majestic Blog.| Majestic Blog
