UBB.threads PHP Forum Software Community Forums UBB.threads 7 Importer Support Rick - Importer Speed for Large Site?

Rick,

We have been trying to get everything laid out to do the upgrade from 6.5 to 7.1. My 2 production servers (1 for HTTP and 1 for MySQL) are both dual-Xeon 2.4s with 2 gigabytes of memory and fast hard drives. My test servers are also dual-processor machines, except with 933s and 1 gigabyte of memory each. On my test servers, the importer script took almost exactly 24 hours to complete. That is a LOT of downtime for a site as active as ours. What can we do to allow for a faster conversion?

I thought about doing the import with the forums online, but that would lose any posts that were made during the import. I also tried the modification that did more rows per refresh, but it did not seem to work for me.

We archive posts ever so often, so our active database is not extremely large, but still pretty sizable. The stats are:

Total members: 74330
Total forums: 36
Total topics: 141011
Total posts: 1180436

Any help is appreciated with making the upgrade as smooth as possible for our users.

Thanks,
Jason

24 hours does seem a little long - I think it took around 12 hours to import with just over 1 million for us.

I simply did it overnight - but kept the old forums up with a notice that said any posts made after say 22:00 would not appear on the new upgraded forums.

I'm working this problem right now. I have an importer from 6.5.5 to 7.1 that did 4,000,000 posts in something like 19 hours. Unfortunately it had a minor bug which I'm fixing now (and then I'll have to test, etc.) My plan is to make it available to the community.

When I tried to import this same table using the normal importer I gave up after 3 days of it running on an empty server.

So was the database on a different server than the importer? If so, I think that could cause the slow import times since the importer is pretty much 30mill queries in your case

Nope. Same server. Just a lot of inserts and updates.

Mine is on a different server.

Jason

I have the same problem.
That's why I still cannot decide when to upgrade from 6.5.4 to 7.0.2.

I have :
Total topics: 84897
Total posts: 851749

On my test server , the importer script has already run 20+ hours.
And the importer script is running more and more slowly. It is processing 64???th topic now.
I estimate that it'll take the whole two days to finish "topic" import. Not including PM's import , that may be another two or three days ...

My question : Why is the script running more and more slowly ?
At first , it processes 20 topics like fly , but later , it's like a snail... and my HD read/write more and more constantly.

I can't shut down my website for 4 or 5 days to upgrade . Is there any better solution ?

How about the ubb6's db and ubb7's db on different servers , will it help ?

Mine slows down as it goes too. I suppose because the tables and indexes keep growing.

Jason

Hmm, there was an issue with the importer for 7.0.x where it would slow down as it goes, that was due to a bad query. Are you running the very latest importer and importing into 7.1?

No, I am running 7.0.2 importer. I think I have to upgrade from 6.5.4 -> 7.0.2 -> 7.1
Is it possible to upgrade direct from 6.5.x to 7.1 ? (skip 7.0.x)

Yes, you can upgrade directly from 6.5.x to 7.1. Just grab the latest importer from the members area as it has some changes that should speed things up during the posts stage as well.

I was running the earlier one as well.

Jason

You can upgrade from 6.5 to ANY release in the 7 series; upgrading to 7.0.2 is not required at all.

Again, i really need to make sure theres not more pages after I decide to post on something...

I grabbed the importer script from members area last night , and ran the importer again.
But I didn't notice any speed improvement till now. It seems the same slow as before ...