UBB.threads PHP Forum Software Community Forums General Suggestions and Feedback Google not crawling my site

HELP! When I do this in google:

site:www.britishcarforum.com/bcforum/ubbthreads.php/ubb/showflat/

It only returns 3 results and the ONLY post that is actually there is one that is accessable from out on my portal page. NONE of my other "threads" seem to be indexed by Google. What the heck is going on? I see a single Googlebot on my site, but It seems that none on my threads are getting indexed - ZERO, ZIP, NADA. I must figure this out! Help!

Basil - just do site:www.britishcarforum.com/bcforum

I notice that google crawl my site onmly with one indexer (showed in 7.1 as spider) and several other bots from google come as images.google... the google images spider.

site:http://www.britishcarforum.com/bcforum
Results 1 - 10 of about 3,220 from www.britishcarforum.com/bcforum. (0.79 seconds)

That's honestly not too shabby; a nice firm base of crawling.

I've go over 27,000 threads and when I look at the stuff that is indexed it dosn't look like many threads.

Ok i see google stuff often on my spider list..

I have the spider friendly thing turned on..

there is only 7 listing of clubadventist.org in google..

I used to have 252,000...

As I watch google, I have concluded that it goes into the calendar but never comes out...

Does that make sense?

here is the only place it visits... am I alone in this problem?

I've checked on all the forums I have admin access to and don't see them in the calendar currently. You might just try adding a robots.txt file to exclude the calendar section.

what is the code for that please...

User-agent: *
Disallow: /UrlTo/ubbthreads.php/ubb/calendar
Disallow: /UrlTo/ubbthreads.php?ubb=calendar
Disallow: /UrlTo/ubbthreads.php?ubb=showday

*should* do it

I think a robots.txt option such as:


Should stop all bots from accessing you calendar

note to self, check next page

hehehe
I so resemble that..

Thanks for the tip(s)

Will see if I can get more than 7 listings in google now

i just blocked the calendar on mine, i wonder if it'll speed up results lol... then again, i use my sitemap addon so that could attribute to why all of my threads are there.

Please splain the site map thing? Is it a one-time deal or do you have to re-do it or update it and if so how often?

A site map is a mapped set of links that google gets from a predefined URL...

A better explanation is a working copy of my sitemap generator:
http://www.elitewebgamers.com/ubb.sitemap.php?type=1&se=2

Basically it grabs all of the id's from your database and populates links for google of what content is where; it assists in the discovery of new urls for google.

You can read more about it on ubbdev here

So let me see if I got this - the site maper generator was used to generate this page:

http://www.elitewebgamers.com/ubb.sitemap.php?type=1&se=2

Then you tell Google that that is your site map? If you have tens of thousands of posts, wouldn;t that file get very big? Or does it not actually list every single thread?

Basil

The file is generated when it's called each week by google; the files size is a constant of under like 8k. The links are built when it's called each week by google's sitemap crawler.

But you have the rest, basically google hits your map and grabs the links, and lists them in its discovery database

How is that different than it just crawling the links themselves?

Google crawls a site, link by link, to discover new links.

A SiteMap feeds them those links directly.

The benefit: You don't have to wait for their bot to discover new threads, you're cutting out 90% of the wait to discover new threads; the mod listing at ubbdev has google's "why should i use a sitemap" article from their help site quoted on it for more information.

It's the difference between you telling them where to look and you hoping they find it on their own. Which one is more certain?