Subject: Re: blocking bots
To: None <email@example.com>
From: Wolfgang Rupprecht <firstname.lastname@example.org>
Date: 01/06/2001 17:28:20
email@example.com (Erik Huizing) writes:
> I noticed yesterday that someone (www.webtop.com) is indexing my server. I
> was wondering if there's any way to put a stop to this. I don't have any
> sensitive information on my server, but its more of an annoyance that this
> is happeneing (all they're really getting is a copy of my apache
> I've tried blocking them in my ipf.conf:
> block in quick on ep0 from 188.8.131.52/24 to any
> (this is the first rule in the file)
> I've also tried specifically blocking port 80, and tcp protocol, and the
> full /32 address, all to no avail.
$ host6 184.108.40.206
220.127.116.11: IPv4 18.104.22.168 '' 'adsl-rg-2-1.pops.easynet.fr'
Are you sure you have that IP address right? You might be being
indexed by a spammer on an adsl line.
I have this in my ipf.conf and can verify that it still works as of an
two hours ago.
block in log quick from 22.214.171.124/32 to any
Jan 6 15:24:40 capsicum ipmon: 15:24:39.930863 tlp0 @0:9 b authorized-scan1.security.home.net,45216 -> c460058-a.frmt1.sfba.home.com,nntp PR tcp len 20 10240 -R IN
On the other hand, I tend to treat web-crawling indexers as friendlies
as long as they wait a few seconds between requesting new pages. Its
only the isp's scanners that I bother to block. (I no longer know if
the clueful people are still running the show at my ISP or if they
have all vested and left.)
Wolfgang Rupprecht <firstname.lastname@example.org>
Coming soon: GPS mapping tools for Open Systems. http://www.gnomad-mapping.com/