Google not crawling my site - UBB.threads PHP Forum Software Community

You are not logged in. [Log In] UBB.threads PHP Forum Software Community » Forums » UBB.threads News and Discussions » Feedback & Suggestions » Google not crawling my site

Site Links

Home
Features
Documentation
Pricing & Order
Members Area
Support Options
UBBDev.com
UBBWiki.com

Top Posters (30 Days)

Ruben	38
Bert	36
Gizmo	23
sb	5
hema0359	4
After the Rose	4
gladiator	3
BellaOnline	3
skicomau	3
matrixinfologic	2

Latest Photos

Page 18 of 19

...

Topic Options

#179260 - 02/23/07 12:51 PM

Re: Google not crawling my site [Re: Stan]

AllenAyres Offline

Registered: 12/29/03
Posts: 2020
Loc: Texas

User-agent: *
Disallow: /UrlTo/ubbthreads.php/ubb/calendar
Disallow: /UrlTo/ubbthreads.php?ubb=calendar
Disallow: /UrlTo/ubbthreads.php?ubb=showday

*should* do it $\:\)$

_________________________
- Allen
- ThreadsDev | PraiseCafe

Top

Express Hosting

Express Hosting "We are the official hosting company of UBB.threads. Ask us about our free migration services to migrate your UBB.threads installation."

#179279 - 02/23/07 04:39 PM

Re: Google not crawling my site [Re: Stan]

Gizmo

Registered: 06/05/06
Posts: 15475
Loc: Portland, OR; USA

I think a robots.txt option such as:

Code:

User-agent: *
Disallow; /forums/ubbthreads.php?ubb=calendar
Disallow: /forums/ubbthreads.php/ubb/calendar
Disallow: /forums/ubbthreads.php?ubb=showday
Disallow: /forums/ubbthreads.php/ubb/showday

Should stop all bots from accessing you calendar

_________________________
Forums: UGN Security & VNC Web Design & Development
UBB.Threads: UBB.Wiki, My UBBSkins, UBB.Sitemaps
Longtime Supporter, Beta Tester & Resident Post-A-Holic.
Modifications, Upgrades, Styling, Coding Services, Disaster Recovery, and more!

Top

#179280 - 02/23/07 04:39 PM

Re: Google not crawling my site [Re: Gizmo]

Gizmo

Registered: 06/05/06
Posts: 15475
Loc: Portland, OR; USA

note to self, check next page

Top

#179283 - 02/23/07 04:48 PM

Re: Google not crawling my site [Re: Gizmo]

Stan

old hand

Registered: 06/05/06
Posts: 886

Originally Posted By: Gizmo

note to self, check next page

hehehe
I so resemble that..

Thanks for the tip(s)

Will see if I can get more than 7 listings in google now $\:\)$

Top

#179289 - 02/23/07 05:07 PM

Re: Google not crawling my site [Re: Stan]

Gizmo

Registered: 06/05/06
Posts: 15475
Loc: Portland, OR; USA

i just blocked the calendar on mine, i wonder if it'll speed up results lol... then again, i use my sitemap addon so that could attribute to why all of my threads are there.

Top

#179313 - 02/23/07 07:25 PM

Re: Google not crawling my site [Re: Gizmo]

Basil

addict

Registered: 08/18/06
Posts: 689
Loc: Southwest US

Please splain the site map thing? Is it a one-time deal or do you have to re-do it or update it and if so how often?

Top

#179315 - 02/23/07 07:44 PM

Re: Google not crawling my site [Re: Basil]

Gizmo

Registered: 06/05/06
Posts: 15475
Loc: Portland, OR; USA

A site map is a mapped set of links that google gets from a predefined URL...

A better explanation is a working copy of my sitemap generator:
http://www.elitewebgamers.com/ubb.sitemap.php?type=1&se=2

Basically it grabs all of the id's from your database and populates links for google of what content is where; it assists in the discovery of new urls for google.

You can read more about it on UBBDev here

Top

#179483 - 02/25/07 08:37 PM

Re: Google not crawling my site [Re: Gizmo]

Basil

addict

Registered: 08/18/06
Posts: 689
Loc: Southwest US

Originally Posted By: Gizmo

So let me see if I got this - the site maper generator was used to generate this page:

http://www.elitewebgamers.com/ubb.sitemap.php?type=1&se=2

Then you tell Google that that is your site map? If you have tens of thousands of posts, wouldn;t that file get very big? Or does it not actually list every single thread?

Basil

Top

#179484 - 02/25/07 08:43 PM

Re: Google not crawling my site [Re: Basil]

Gizmo

Registered: 06/05/06
Posts: 15475
Loc: Portland, OR; USA

The file is generated when it's called each week by google; the files size is a constant of under like 8k. The links are built when it's called each week by google's sitemap crawler.

But you have the rest, basically google hits your map and grabs the links, and lists them in its discovery database