Subject: Re: blocking bots
To: Ty Sarna <firstname.lastname@example.org>
From: Alex van Denzel <email@example.com>
Date: 01/24/2001 23:52:33
Ty Sarna wrote:
> > Have you tried putting a robots.txt file in place to prohibit
> > webcrawlers? That's the high-level solution to try first.
> And it should get all of them, without you having to know IPs or
Yeah. For benign robots. Malicious robots ignore the file and happily
create server load, scan for e-mail addresses to spam, etc.
Greetings (normally not so pessimistic)
Replace bitbucket with anything else.