UBB.threads PHP Forum Software Community Forums UBB.classic [Archived] UBB.classic Bugs [NOTABUG] Spiders spidering board but not retrieving content...

Ok, I run a nice sized bulletin board over at UGN, before the upgrade to 6.7 we were using a spider addon (one of the first main ones). Before upgrading we had about 6000+ results in google from the url:

http://www.google.com/search?hl=en&ie=UT...G=Google+Search

However I just checked it and I'm down to a little over 2000... Basically what I'm saying is that the spider capability for UBB.Classic doesn't work.

My URL's are showing as:
http://www.undergroundnews.com/ubb/ultimatebb.php?/topic/26/60.html#000011

Where on UBBCentral, UBBDev their showing as:
http://www.undergroundnews.com/ubb/ultimatebb.php/topic/26/60.html#000011

Now I've confirmed this at AlForAdmin's forums with:
http://www.google.com/search?hl=en&lr=&i...G=Google+Search

I've verified that his board has had spiders on it through WOL.

I have the following spider settings set in my CP:
URL Type: Short spider-friendly URLs
Use fake ".html" extensions for spider-friendly URLs?: on
Use Spider-friendly URL Compatabiltiy Mode?: on

And yes I know that modified bulletin boards aren't covered under warrenty, however my unmodified bulletin board at:
http://www.promodtecnologies.com/ is also experiancing the same issue.

Keep in mind that do to ISE errors, I had google remove all of my pages from their site a week ago. I'm not actually supposed to have any pages listed for a month or so

Ok fine, both of my boards, which were extremely popular in google, have nothing to show after this change... Yet google and inktomi both hit my board every so often (i can tell from the WOL admin view).

Chances are that the old spider hack did not use the same URL structure as the new URLs, thus breaking all the links. It will probably take some time for Google and Inktomi/Yahoo to pick up the new URL format...

But, their spidering the site nightly, and they have no results... One whould think that at least there would be something to show becides my main board page...

Yeah, that's true.

However, I don't see why they shouldn't be getting any content...

CC, could it have anything to due with the ?/ ?

It's the only thing Gizzy and I have in common, Safe-Mode


My spidering for March. I haven't had a single page indexed since I upraded to 6.7

In my tests, the spiders didn't care about the query string, and spidered anyway. Apparently that changed at some point during development...

Hello,

Experiencing the same issue on our server. Google seems to not spider the forums at all. We have the same config as Gismo's.

Med.

Med, have a link?

Gizmo -

I think it started to crawl it. But seems like it took only the first pages of forum &...mmm calendar!

Thx

Can I see a link to your forums? via PM or here, whichever you choose...

I found an issue with the Ensim management system that some hosts use and a fix for it (in most cases, unfortunately my site is one of the ones it didn't work on lol)...

A little update: I was able to get my awesome host to change my PHP settings so that I could diable compatability mode and I now have 500+ pages listed on google.

Hi... hmmm... not spidering anymore

http://www.amabilia.com/cgi-bin/noncgi/ultimatebb.php

What was the issue you were talking about Gizmo ?

Thanks for the reply,

Yes...
http://www.amabilia.com/cgi-bin/noncgi/ultimatebb.php?/forum/2.html

Google sees the ? and turns away. It's the exact issue lol... It's funny though, cas if you can't have compatability off then theres no point to using spiderable url's :/...

/me likes the short urls even if they aren't spidered.

Gizmo-

And how do you change this ? Thx

Med

Login to your CP, under "Primary Settings" choose Advanced and detick "compatability mode".

Also, if the page spews an ISE, could you email your host and ask if they use Ensim? If they do I may have a fix for you...

Gizmo-

Yes... 404 nearly every page. I'll see with my host.

Thx again,

Med

Gizmo-

What would be the link between these 2 ?

and how come sometime the ? is at the end and sometime in the middle.

Thx

Med.

404? it should be a 500 ISE. If it's a 404 theirs something with the mod's you're running as 404 means cannot be found.

I have no clue about the positioning of the ?, it's in the code.

Yes... 404s. It's even written on the compatibility ticker "Enable this option if your server returns 404 errors when spider-friendly URLs are enabled."

Mmmmm.. no clue why this not working.

Med.

Very odd; the only fix i have is for ensim issues (which usually give out ISE's) lol... Do you have any board mod's installed?