UBB.threads PHP Forum Software Community Forums Resources and Customizing Admin Zone Spiders

I've noticed a dramatic decline in spider activity lately. Has anyone else seen this, or do I need to do something? I'm using the spider list that Ruben (and then Gizmo) posted several months ago.

As long as the important ones are still there, that's the main thing. I think Yahoo has cleaned things up a bit. We used to have 20-30 Yahoo spiders showing up on the list, but it appears that now they've got it so it only shows up once (keeps coming from the same IP), instead of 30 different ones.

You are probably correct. I haven't been paying attention to which spiders were crawling - just the total number.

Earlier this month, my site was hit with a 2-3 fold jump in traffic. Ran the hit count and bandwidth numbers really high. Then suddenly one day, they dropped back to normal.

I don't know if it was undeclared crawler activity or what.