the sitemap generator is going on forever, it's reflecting the same problem so i'm stopping it at almost 350,000 files after over 24 hours. (most large sites of say 10,000 files take 20 minutes usually)
Here is the current plan until Rick can change this. I shall:
1) Study the URLs being generated by UBB.
2) Study the SQL DB tables used by UBB
3) Write a script that,
...a) Does a SQL query to get a list of topics direct from the DB
...b) For each topic in SQL build a unique URL
...c) Output the list to XML and a HTML file.
4) In the spider, set the start point to this HTML file.
i can also use the same XML file as part of a Google site map. The result can't be infinite if i get the list from the SQL DB. This is a serious coding job however.
