ok, i have solved this. 3 weeks and $600 in bandwidth later

The $50 i paid Gizmo to create a sitemap generator for my board saved the day (thanks gizmo).

it crawled the board and got just under 24,000 threads. a very reasonable amount. in a very reasonable amount of time. using a reasonable amount of bandwidth. I'm thrilled.

any other crawlers i used got like infinite, past 600000 once. cost me $50 in bandwidth, for ONE two day crawl. and it was still running.

so if you want to save a lot of time and headaches and want an excellent sitemap of your site in any of the main formats accepted by the major search engines, give gizmo a PM. super fast, he was helping new users tonite on my board, very complete, he knows the ubb software like i know belize. very well.

super duper helpful. he also did a great implementation of the IRC chat. all tonite!!!

here's the chat, new larger chat i call it!

http://ambergriscaye.com/forum/ubbthreads.php/ubb/chat

to login use
username: sample
and
password: sample

to see it. you have to be logged in

and one more big shout box thanks to gizmo.