|
|
Joined: Jun 2006
Posts: 9,242 Likes: 1
Former Developer
|
|
Former Developer
Joined: Jun 2006
Posts: 9,242 Likes: 1 |
650 search engine spiders in the past half hour, wow.
|
|
|
|
|
Joined: Jun 2006
Posts: 9,242 Likes: 1
Former Developer
|
|
Former Developer
Joined: Jun 2006
Posts: 9,242 Likes: 1 |
Make that 665. If this keeps up, we'll break our most online record, that was set only 2 days ago 
|
|
|
|
|
Joined: Jun 2006
Posts: 1,344
veteran
|
|
veteran
Joined: Jun 2006
Posts: 1,344 |
LOL, they sure are crawling. I had 350 the other night
|
|
|
|
|
Joined: Nov 2006
Posts: 3,095 Likes: 1
Carpal Tunnel
|
|
Carpal Tunnel
Joined: Nov 2006
Posts: 3,095 Likes: 1 |
672 Search Spiders right now
|
|
|
|
|
Joined: Jul 2006
Posts: 4,057
|
|
Joined: Jul 2006
Posts: 4,057 |
680 How can you possibly have this many ?
BOOM !! Version v7.6.1.1
People who inspire me Isaac ME Gizmo
|
|
|
|
|
Joined: Jun 2006
Posts: 3,837
Carpal Tunnel
|
|
Carpal Tunnel
Joined: Jun 2006
Posts: 3,837 |
Aren't you the lucky ones  LOL
|
|
|
|
|
Joined: Jul 2006
Posts: 4,057
|
|
Joined: Jul 2006
Posts: 4,057 |
693 Search Spiders. here. repost but more relevant to this topic nope 704 Search Spiders. nope 706 Search Spiders.
Last edited by Mark S; 06/21/2007 2:28 PM.
BOOM !! Version v7.6.1.1
People who inspire me Isaac ME Gizmo
|
|
|
|
|
Joined: Jan 2004
Posts: 2,474 Likes: 3
Pooh-Bah
|
|
Pooh-Bah
Joined: Jan 2004
Posts: 2,474 Likes: 3 |
I've been getting the same thing recently.
Heck of a lot of spider activity.
My site got a new record at 6.04am the other day.
I didn't see the details, but I estimate there were 500+ spiders online.
(My site is 90% UK - who's online at that time?)
|
|
|
|
|
Joined: Jul 2006
Posts: 4,057
|
|
Joined: Jul 2006
Posts: 4,057 |
TAMU.edu - Search spider that's a new one searching my forums
BOOM !! Version v7.6.1.1
People who inspire me Isaac ME Gizmo
|
|
|
|
|
Joined: Jan 2004
Posts: 2,474 Likes: 3
Pooh-Bah
|
|
Pooh-Bah
Joined: Jan 2004
Posts: 2,474 Likes: 3 |
Yes, I have that one also.
Just one instance now and again though - not exactly a Yahoo competitor.
|
|
|
|
|
Joined: Jul 2006
Posts: 4,057
|
|
Joined: Jul 2006
Posts: 4,057 |
Fancy a American University creaming info from our site lol
BOOM !! Version v7.6.1.1
People who inspire me Isaac ME Gizmo
|
|
|
|
|
Joined: Jan 2004
Posts: 2,474 Likes: 3
Pooh-Bah
|
|
Pooh-Bah
Joined: Jan 2004
Posts: 2,474 Likes: 3 |
Actually, I've just checked and I have 250(ish) spider - all Yahoo.
What do you think has brought on this sudden frenzied spidering?
|
|
|
|
|
Joined: Jul 2006
Posts: 4,057
|
|
Joined: Jul 2006
Posts: 4,057 |
BOOM !! Version v7.6.1.1
People who inspire me Isaac ME Gizmo
|
|
|
|
|
Joined: Jan 2004
Posts: 2,474 Likes: 3
Pooh-Bah
|
|
Pooh-Bah
Joined: Jan 2004
Posts: 2,474 Likes: 3 |
Following Marks insightful comments on the content of my site (ahem!) I did a search for the word porn. And yes - there were 2 results... - One regarding a Digital inflection rigidometer. Users view porn to get 'started'
- The other when a competitor website tried to trash my site by saying that it was affiliated to a porn site. (One of my members was on the committee of a website that became defunct and their forum was overrun with porn spammers.
Check your boards - do you have 'porn' on them? 
|
|
|
|
|
Joined: Jul 2006
Posts: 4,057
|
|
Joined: Jul 2006
Posts: 4,057 |
OMG
I have 8 pages "Max" limit reached lol
But i have topics called NOT PORN
And a game called Porn Star name too lol
BOOM !! Version v7.6.1.1
People who inspire me Isaac ME Gizmo
|
|
|
|
|
Joined: Jan 2004
Posts: 2,474 Likes: 3
Pooh-Bah
|
|
Pooh-Bah
Joined: Jan 2004
Posts: 2,474 Likes: 3 |
Rose is going to be very upset to hear of this Mark. She thought you were sooooh sweet Showing your true colours now, mate 
|
|
|
|
|
Joined: Jul 2006
Posts: 4,057
|
|
Joined: Jul 2006
Posts: 4,057 |
Its not me it my users lol
And its all clean lol
Rose will understand, she has too or life just aint worth living lol
The game 3 words after sex don't help either
BOOM !! Version v7.6.1.1
People who inspire me Isaac ME Gizmo
|
|
|
|
|
Joined: Nov 2006
Posts: 3,095 Likes: 1
Carpal Tunnel
|
|
Carpal Tunnel
Joined: Nov 2006
Posts: 3,095 Likes: 1 |
The game 3 words after sex don't help either Maybe you don't know the 3 correct words. YOU'RE RIGHT DEAR
|
|
|
|
|
Joined: Apr 2007
Posts: 3,940 Likes: 1
Former Developer
|
|
Former Developer
Joined: Apr 2007
Posts: 3,940 Likes: 1 |
hrm. technically that's 4
(I KEED)
|
|
|
|
|
Joined: Nov 2006
Posts: 3,095 Likes: 1
Carpal Tunnel
|
|
Carpal Tunnel
Joined: Nov 2006
Posts: 3,095 Likes: 1 |
Not really, it's a contraction for you are
So, debatable on many sites (not an issue I care to debate though)
|
|
|
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116 |
456 se spiders; all but 1 being yahoo lol
|
|
|
|
|
Joined: May 2006
Posts: 579
addict
|
|
addict
Joined: May 2006
Posts: 579 |
I had two pages of it. But only used in the most innocent of ways. Speaking of search bots, I have a question. I don't really get many search bots - maybe four or five at a time. I've always thought it was because I don't have the following checked in the CP. Enable Spider-friendly URLs? Warning: This function allows search engine spiders to crawl the community, which may result in higher bandwidth usage and server load. Note: The PATH_INFO environmental variable must be available for this feature to function properly. The reason I don't have this checked, is because I don't want individual posts coming up in search results in Google and other engines. Do I have a correct understanding of this function? Just wondering.
|
|
|
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116 |
SE Friendly urls make it "easier" for SE's to troll your board, yes; though not impossible
|
|
|
|
|
Joined: May 2006
Posts: 579
addict
|
|
addict
Joined: May 2006
Posts: 579 |
But am I right in thinking that if I have that box checked, individual posts will appear in search engine results?
|
|
|
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116 |
All that box will do is help SE's spider pages easier; they can spide without the option.
And if pages are included in google then yes, individual posts will be in there too.
You'll find that even the most mudayne posts will bring in users from search engines lol
|
|
|
|
|
Joined: Apr 2007
Posts: 3,940 Likes: 1
Former Developer
|
|
Former Developer
Joined: Apr 2007
Posts: 3,940 Likes: 1 |
hrm. so if i post sex slave midget and then a real serious post, google will bring em on in.. note to self.. preface all posts with above words
|
|
|
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116 |
lol, kinda ... Our most popular incoming thread at UGN?? HotChicks thread... Kinda like UBBCentral's longest thread lol
|
|
|
|
|
Joined: May 2006
Posts: 579
addict
|
|
addict
Joined: May 2006
Posts: 579 |
Well, my users are totally against having individual posts listed in search engines, and so am I really. I guess I'll leave that unchecked.
|
|
|
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116 |
well, see, it's not just individual posts; it's the entire topics (and as individual posts are a PART of the topic, they're included as well).
BTW, nothings to say that they won't crawl it anyway, unless you have a robots.txt rule to specificaly tell search engines not to crawl your forums.
|
|
|
|
|
Joined: May 2006
Posts: 579
addict
|
|
addict
Joined: May 2006
Posts: 579 |
I've tried searching for posts on Google, and they don't come up. People talk about some real personal stuff on my forums, and I'd really hate to see those come up in search results.
I do have a robots.txt, but I don't restrict the forums.
|
|
|
|
|
Joined: Apr 2007
Posts: 3,940 Likes: 1
Former Developer
|
|
Former Developer
Joined: Apr 2007
Posts: 3,940 Likes: 1 |
do you restrict anonymous users? ie: are they forced to sign up to see forum info?
|
|
|
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116 |
To protect your forums, if you actually don't want SE's crawling things, you should add this to your robots.txt... User-agent: *
Disallow: /forums
|
|
|
|
|
Joined: May 2006
Posts: 579
addict
|
|
addict
Joined: May 2006
Posts: 579 |
SD, anonymous users can read the boards, but they can't post without registering. Why do you ask?
Giz, I allow them to crawl the forums - in fact I want the forums to be listed in the search engines. I just don't want individual posts to be crawled. I thought that by not checking that checkbox, it prevented that. I could be right though - because I haven't seen an individual post from my forums in a search listing since I've had UBB.
|
|
|
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116 |
nonono... ok...
REGUARDLESS of if you have SE friendly enabled or not, you will get crawled. SE links just help the process along and streamline the crawling of content.
If THREADS are indexed, invididual posts are as well, as posts are a PART of the thread...
The content of which the search engines use to bring you new users is from the individual posts.
If you WANT your site crawled by SE's you should enable SE frinedly URLs to help the process, though by not enabling them you're still allowing se's to crawl, it'll just be more difficult for them to do so...
|
|
|
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116 |
BTW, some of your posts/threads are in google: clicky clicky
|
|
|
|
|
Joined: May 2006
Posts: 579
addict
|
|
addict
Joined: May 2006
Posts: 579 |
Yikes!
Ok .. but who will ever use that search term? I took a phrase out of one of those posts, and put it into Google, and it didn't come up in the results list.
So, the REASON for that check box, is to make it easier for bots?
|
|
|
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116 |
It won't nessessarily return just any term for just any search; it's kind of hard to explain briefly..
Well, the "reason" for the box is to clean up how URL's look, and how url's are fed to the browser/client requesting them; it's just that the "cleanups" work well with search engines; the initial cleanups in previous code where based off of proven techniques...
If you surf around UBBDev about 2-3 years ago you'll see me talking about search engine spiders and a paid search engine hack that I was using at the time; this was just before SE friendly url's where introduced into the feature set (of at least classic; I don't know when they where put into threads as I wasn't a threads6 user).
|
|
|
|
|
Joined: May 2006
Posts: 579
addict
|
|
addict
Joined: May 2006
Posts: 579 |
Thanks Giz .. I understand now. I've always found SE stuff difficult to understand, although I recently went right through the Google Webmaster tools, doing everything they recommended for ratings and exposure. I am quite serious about not having individual posts appear in listings, although what I thought was preventing that looks as though it might be incorrect. I might do as you suggest, Giz, and use the robots.txt to prevent the reading of individual forums, but then activate that checkbox. Is that what you'd recommend?
|
|
|
|
|
Joined: Dec 2006
Posts: 1,235
veteran
|
|
veteran
Joined: Dec 2006
Posts: 1,235 |
hrm. so if i post sex slave midget and then a real serious post, google will bring em on in.. note to self.. preface all posts with above words Or use the "invisible post footer" option. Left click and drag from here > sex slave midget hotchicks < to here. 
|
|
|
|
|
Joined: Jul 2006
Posts: 4,057
|
|
Joined: Jul 2006
Posts: 4,057 |
quality i like that trick
BOOM !! Version v7.6.1.1
People who inspire me Isaac ME Gizmo
|
|
|
|
|
Joined: Apr 2007
Posts: 3,940 Likes: 1
Former Developer
|
|
Former Developer
Joined: Apr 2007
Posts: 3,940 Likes: 1 |
color=FFF shows up perfectly in the dark theme i'm using, so it's really not invisible.. but i get your drift
|
|
|
|
|
Joined: May 2006
Posts: 579
addict
|
|
addict
Joined: May 2006
Posts: 579 |
I think it does, driv. Thank you. But given that each forum page is dynamically built, where would I put the meta tags?
|
|
|
|
|
Joined: Jul 2006
Posts: 4,057
|
|
Joined: Jul 2006
Posts: 4,057 |
Control Panel » Display Options >>>> HTML Includes Header Insert: Use for meta tags, etc. That should cover every page
BOOM !! Version v7.6.1.1
People who inspire me Isaac ME Gizmo
|
|
|
|
|
Joined: Jan 2004
Posts: 2,474 Likes: 3
Pooh-Bah
|
|
Pooh-Bah
Joined: Jan 2004
Posts: 2,474 Likes: 3 |
What a team!!!!
|
|
|
|
|
Joined: May 2006
Posts: 579
addict
|
|
addict
Joined: May 2006
Posts: 579 |
You guys are awesome! So, it looks to me as though I CAN have it both ways. Big hugs to you all.
|
|
|
|
|
Joined: May 2006
Posts: 579
addict
|
|
addict
Joined: May 2006
Posts: 579 |
So, I'm not quite sure about the difference between the NOARCHIVE and NOINDEX tags. They sorta sound the same to me.
|
|
|
|
|
Joined: Jan 2004
Posts: 2,474 Likes: 3
Pooh-Bah
|
|
Pooh-Bah
Joined: Jan 2004
Posts: 2,474 Likes: 3 |
Check out the last paragraph of that page... Also, the NOARCHIVE tag controls only whether the cached page is shown. To prevent the page content from being indexed, use the NOINDEX tag.
|
|
|
|
|
Joined: Dec 2006
Posts: 1,235
veteran
|
|
veteran
Joined: Dec 2006
Posts: 1,235 |
NOINDEX - tells search engine not to index a specific page
NOARCHIVE - tells search engine not to store a cached copy of your page
NOFOLLOW - tells search engine not to follow the links on a specific page
|
|
|
|
|
Joined: Jan 2004
Posts: 2,474 Likes: 3
Pooh-Bah
|
|
Pooh-Bah
Joined: Jan 2004
Posts: 2,474 Likes: 3 |
Like I said What a team!!!! I luv quoting meself - makes me feel important in my own small world...... 
|
|
|
|
|
Joined: Jul 2006
Posts: 4,057
|
|
Joined: Jul 2006
Posts: 4,057 |
don't do your post count any harm either And its over 15 characters lol
BOOM !! Version v7.6.1.1
People who inspire me Isaac ME Gizmo
|
|
|
|
|
Joined: Jan 2004
Posts: 2,474 Likes: 3
Pooh-Bah
|
|
Pooh-Bah
Joined: Jan 2004
Posts: 2,474 Likes: 3 |
don't do your post count any harm either And its over 15 characters lol Now then - there's a thought. But what is 15 chars? Is this fifteen ??? (It has 2 spaces) Is this 15 ? ??? (It has 4 spaces a question mark and a smiley)
|
|
|
|
|
Joined: Jul 2006
Posts: 4,057
|
|
Joined: Jul 2006
Posts: 4,057 |
seeing that you asked me question mark and a smiley I dunno that's why i follow up with crap lol <<< Opps nearly forgot the smiley
BOOM !! Version v7.6.1.1
People who inspire me Isaac ME Gizmo
|
|
|
|
|
Joined: May 2006
Posts: 579
addict
|
|
addict
Joined: May 2006
Posts: 579 |
Ok, what does the following mean for the enable spider-friendly URLs?
Note: The PATH_INFO environmental variable must be available for this feature to function properly.
|
|
|
|
|
Joined: May 2006
Posts: 579
addict
|
|
addict
Joined: May 2006
Posts: 579 |
I'm guessing it's significant, because I selected that checkbox, and my main page banner disappeared! Yikes!
|
|
|
|
|
Joined: Apr 2007
Posts: 3,940 Likes: 1
Former Developer
|
|
Former Developer
Joined: Apr 2007
Posts: 3,940 Likes: 1 |
lol Rose. it's fun just following your travails !
keep going!
|
|
|
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116 |
Well, if you add the robots.txt option that i specified, it'd completely block SE's from transversing your forums, which would be th eonly way to block them from trolling any of the content on your forums. SK, sweet trick
|
|
|
|
|
Joined: Jan 2004
Posts: 2,474 Likes: 3
Pooh-Bah
|
|
Pooh-Bah
Joined: Jan 2004
Posts: 2,474 Likes: 3 |
I'm guessing it's significant, because I selected that checkbox, and my main page banner disappeared! Yikes! I can't see ANY reason for that happening. I suspect you have some sort of 'local' issue? Give it an hour or two - I'm sure it will resolve itself
|
|
|
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116 |
color=FFF shows up perfectly in the dark theme i'm using, so it's really not invisible.. Lol; should have set the background property (I think we have that option hidden away somewhere)... As for meta tags, these seem to do about the same thing as the robots.txt thing; only the robots.txt thing will save you bandwidth as they won't even bother trying to transverse your space. I luv quoting meself - makes me feel important in my own small world...... Whatever gets you through the night
|
|
|
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116 |
don't do your post count any harm either And its over 15 characters lol good old mark, looking out for everyones postcount... Ok, what does the following mean for the enable spider-friendly URLs?
Note: The PATH_INFO environmental variable must be available for this feature to function properly. PATH_INFO is an enviroment variable which allows the UBB to read the "miscelanious text" after ubbthreads.php; if you don't have this enabled on your server, you can't use short URLs. Theres an option on the FAQ which covers this and ways to fix it. lol Rose. it's fun just following your travails !
keep going! the wobulous travels of Rose lol... I do however find it odd that rose's header dissapeared, i can't see why that would happen from enabling shorturls...
|
|
|
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116 |
did you know that the character limit counts quoting users too? :x...
|
|
|
|
|
Joined: Jan 2004
Posts: 2,474 Likes: 3
Pooh-Bah
|
|
Pooh-Bah
Joined: Jan 2004
Posts: 2,474 Likes: 3 |
Didn't really mean it as a post-count-boost just a bit of banter, really.
|
|
|
|
|
Joined: Jul 2006
Posts: 4,057
|
|
Joined: Jul 2006
Posts: 4,057 |
did you know that the character limit counts quoting users too? :x...
BOOM !! Version v7.6.1.1
People who inspire me Isaac ME Gizmo
|
|
|
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116 |
did you know that the character limit counts quoting users too? :x... EXACTLY!
|
|
|
|
|
Joined: Jul 2006
Posts: 4,057
|
|
Joined: Jul 2006
Posts: 4,057 |
BOOM !! Version v7.6.1.1
People who inspire me Isaac ME Gizmo
|
|
|
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116 |
It's all just a trick, a really, lame, trick...
|
|
|
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116 |
btw, i'm pretty sure rick's calculator is based off of characters, may they be a smiley or not... "Hey, this is cool !" would be 21 characters (well, 23 if you include the "'s)
|
|
|
|
|
Joined: Jun 2006
Posts: 464 Likes: 1
Addict
|
|
Addict
Joined: Jun 2006
Posts: 464 Likes: 1 |
LOL in about 2 weeks a startup site I have is already posting at number 1.
Search "Warhammer of Destiny" in google.
I am just about ready to drop the one site and release this new site.
Happy Customer !!!
|
|
|
|
|
Joined: Jul 2006
Posts: 4,057
|
|
Joined: Jul 2006
Posts: 4,057 |
yeh but a true test is without ""
BOOM !! Version v7.6.1.1
People who inspire me Isaac ME Gizmo
|
|
|
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116 |
<snicker> a sitemap would do you wonders mors :x
|
|
|
|
|
Joined: Jun 2006
Posts: 464 Likes: 1
Addict
|
|
Addict
Joined: Jun 2006
Posts: 464 Likes: 1 |
Gizmo has had 673 posts in the last 2 hours !!
Happy Customer !!!
|
|
|
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116 |
lol, no, i'd go 100 or so in the last 24 though :x
|
|
|
|
|
Joined: Jun 2006
Posts: 464 Likes: 1
Addict
|
|
Addict
Joined: Jun 2006
Posts: 464 Likes: 1 |
<snicker> a sitemap would do you wonders mors :x I am buying yours Gizmo. I just haven't been home long enough to send the 50 bucks and be there to install it Next week I will install it. Get ready to send it..
Happy Customer !!!
|
|
|
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116 |
With you and the other inqueries this week, brings the total usercount to 11 I think; gaining momentum :x
|
|
|
|
|
Joined: May 2006
Posts: 579
addict
|
|
addict
Joined: May 2006
Posts: 579 |
I'll have to check it out.
|
|
|
|
|
Joined: Jul 2006
Posts: 4,057
|
|
Joined: Jul 2006
Posts: 4,057 |
650 search engine spiders in the past half hour, wow. I was lucky if i averaged 30 or so search spiders and the occasional peak of 90 until last night at 2:30am 564 <<< Search Spiders WOW No idea how and why lol But enjoying the attention Q - Gizmo lol
BOOM !! Version v7.6.1.1
People who inspire me Isaac ME Gizmo
|
|
|
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116 |
Why, why, why... Well, if you have my sitemap installed the "frequency" is set to daily so they could be visiting back to see if anything is new...
|
|
|
|
|
Joined: Jul 2006
Posts: 4,057
|
|
Joined: Jul 2006
Posts: 4,057 |
if ? You know i have it installed lol Its version 3
BOOM !! Version v7.6.1.1
People who inspire me Isaac ME Gizmo
|
|
|
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116 |
... Was speculating! lol... I think I'm going to add a tick to adjust the default frequency... Or set it so if it was posted after X ammount of days it's adjusted...
|
|
|
|
|
Joined: Jul 2006
Posts: 4,057
|
|
Joined: Jul 2006
Posts: 4,057 |
v4
BOOM !! Version v7.6.1.1
People who inspire me Isaac ME Gizmo
|
|
|
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116 |
<snicker> development of v0.4 will be pushed back till at minimum the 3rd week of July; Ian has some test changes in the one I gave him ...
|
|
|
|
|
Joined: Nov 2006
Posts: 3,095 Likes: 1
Carpal Tunnel
|
|
Carpal Tunnel
Joined: Nov 2006
Posts: 3,095 Likes: 1 |
Well spiders are down by 400 or so now. Still a lot of them here though.
|
|
|
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116 |
414 at UGN right now
Last edited by Gizmo; 07/02/2007 2:32 AM.
|
|
|
|
|
Joined: Nov 2006
Posts: 3,095 Likes: 1
Carpal Tunnel
|
|
Carpal Tunnel
Joined: Nov 2006
Posts: 3,095 Likes: 1 |
If you're using a map wouldn't you want them to die down quite a bit more and only come back periodically and index more?
|
|
|
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116
|
|
Joined: Jun 2006
Posts: 16,299 Likes: 116 |
With some data yes; but it's a forum, so data is constantly changed; so you don't want them to only check back once a week or so as they will not get content.
They also check the pages for new data, if nothing is new, they'll move on to the next pages.
|
|
|
Bots
by Outdoorking - 04/13/2024 5:08 PM
|
|
|
|
|
|
|
|
|
|
|
1 members (1 invisible),
702
guests, and
193
robots. |
|
Key:
Admin,
Global Mod,
Mod
|
|
|
|
