Roast topics
Find topics
Roast it!
Roast topics
Find topics
Find it!
Login
From:
The Boston Diaries
(Uncensored)
subscribe
Why do they even bother with /robots.txt?
https://boston.conman.org/2025/10/01.1
links
backlinks
Tagged with:
robots.txt
web bots
google-cloud-platform
bad web bots
web scrapers
It's still too early to see if limiting IP connections to my webserver is doing any good,
