my 85,000 went a day, and I wish I had thought of alerting webhost. Seems so obvious now.

Webhost:
. Apache and MySQL are going APE!
.. Fix: Restart them both.

User:
. My import keeps stopping
.. Restart it!

Webhost:
. Apache and MySQL are going ape again, apache listing shows http://www.thissite.tld.
.. Fix: Suspend http://www.thissite.tld

I started a test import of a very old version of my database, because I wanted to try it out before I tried it on a newer version and because that copy was already upgraded to 6.5.x.

Almost 20 hours later it's still not finished. It's on topic 26000 out of 33000, and the live database has over 41000 topics, I wonder how long that's going to take. Granted, the live server is quite a bit faster than the test server, but still it seems a bit insane. This is on a Pentium 3 700 MHz with 256MB memory, but the load is around 5% normally (with peaks up to 60%) so it's hardly a performance problem.

Had a look at the code to see if it was easy to change the number of topics per page load, but the numbers were hard coded. It would be nice if it was a variable set at the top so that there is only one place to change. Less risk of mistakes. The load won't be a problem, neither is an overzealous server admin, as it's my own dedicated server and I plan on shutting down the site for a weekend to do the upgrade. Especially since it will take me the better part of a day just to upgrade the database from v6.1 to v6.5...

Is this on Linux or Windows? Flat file transfers are a bit SLOW on most systems. Are you sure RAM and Swap are not being fully utilized?

It's Fedora Linux, there is no significant swap usage. Raised it to 50 topics per page load, but I don't really think it affected the time/topic by much. I'm closing in on 30 hours now and Iä've got 1000 topics to go.

The MySQL isn't tuned, and it's on v4.1.20, but I can't see that it would affect this that much.

It's difficult to say without doing some real analysis.

It could be that the code to do the import is just not well written. It could be your Disks are old and slow. Probably nothing you're going to be able to quickly do except maybe check and see if RAM is an issue. The other items you probably just can't really fix without a lot of time / cost or both.

If your database is 150+MB then you have to be low on memory usage with 256MB for the entire system. If the database is less than that then memory might be okay.

That's definitely slower than normal. The longest import I've had yet has been about 24 hours from a classic board that has 1.1 million posts and about 80k users.

memory *could* come into play if you have a number of topics with a large number of replies as when it imports them, it reads all of the replies of a topic into memory and then writes them to the new database.

The biggest factor is probably disk speed as there are a large number of database queries done on each refresh, so if your disk speed is slow it will definitely lead to a slow import.

It took about 31 hours before it finally finished. Certainly won't do that again on the test server, next time will be on the production server for sure. I haven't checked, but I don't think I have that many really long threads.

Don't really know what kind of disks are in the test server, but the server is quite old so it's probably no speed freaks.

A CLI version of the importer script would be quite nice though, would feel a bit safer than having to rely on the browser, and it wouldn't need to be done in steps either.

Now I'm off to have a look at all the new features and to start tweaking.