Marty,
Ray is away on christmas holidays. So I am doing support for the next couple of weeks.
Before Ray left he discussed this issue with me. And I know he spend several hours researching the problem and looking for a solution.
Our conclusion was that we don't have a good solution for the problem of indexing the new UBB forums that are using the URL re-writing techniques to strip the parameters from the URLs. It is moderately complex issue but I know Ray wrote you an E-mail attempting to explain why it isn't going to work.
In our opinion, the whole idea of re-writing the URL of the UBB forums has been badly thought out and has left the forum being search engine unfriendly (probably the exact opposite of what the UBB people intended).
Zoom filters pages to prevent duplicate pages, this is done based on the URL and content. But obviously there is a near infinite number of unique URLs being generated by the UBB script, and almost as bad, they have made it so that each page is also subtly different from every other page. Thus preventing filtering based on identical HTML content.
I would suggest that you limit your indexing of UBB sites to maybe 5000 pages until UBB correct the problem. It is in the interest of all UBB users to have their forums search engine friendly for Google, Yahoo, etc...
Kind Regards
David Wren
http://www.wrensoft.com
