Spiders?

[david]

Someone at a local Computer Univesity is running a spider on one of my clients sites. No idea why or what they are trying to do but they are burning up over 900mb of bandwidth in just a few days!!

the Moderizer stats show over 90% of traffic is from this one spider and shows 400 online users?? from same IP

Any help in stopping this or ideas why they are doing this would be appreciated.

thanks
David

[pano]

You can use htaccess to deny access to them. You can also deny entire blocks of ip addresses. Place the following in a .htaccess file and place the .htaccess file in each directory you want to deny access:

order allow,deny
deny from 987.65.4.3
deny from 987.65.4.
allow from all

Be sure to replace the ip addresses above! The top one will deny from a specific ip address, the other one will deny from anyone within that ip block...you can use either one. Be careful when denying entire blocks of ip addresses that you don't inadvertantly deny access to the wrong people.


Jim

[admin]

David I saw your ticket in our system we could block them at the router or firewall but what pano is saying would work fine as well.

[david]

Thanks Jim & Admin

the .htaccess seems to have done the trick! But I still cant figure out what they (the spider) were trying to do? Not an email harvester and not a site nabber.

But my stats in oscommerce were showing more 500 simultanous online users or visitors (real figure was just 2) and all I can see was the bandwidth being eaten up? by the gig. Strange!