I have a UBB.classic forum here which uses UTF-8 as the character set.

I'm trying to import this into UBB.threads 7.0.2 using the classic_import script, but all the non-ASCII characters in the posts are being mangled. The MySQL database is set to use UTF-8, and any new posts I make appear to be stored correctly.

How can I convince the importer to insert the posts as UTF-8?

Thanks!

You'll probably need to edit the classic_import.php script. Around line 68 of the script you'll see the header section it spits out for the page. You can add this to it and it *should* do the trick:

<meta http-equiv="Content-Type" content="text/html; charset="UTF-8" />

I was very hopeful - it appeared to do the right thing while importing - the post subjects displayed correctly as they went past which they didn't before - but, sadly, when I went to the forum, the posts were still mangled.

I think that patch fixed the display output of the importer, but not the actual import.

Any other ideas?

You might need to change the language encoding in the generic.php language file to UTF-8 as well.

A good way to test if it actually imported the data correctly is if you have something like phpmyadmin installed on the server. This would allow you to look at the database directly, specifically the ubbt_POSTS table to see if it's an import issue or if it's a display issue.

I checked with phpmyadmin, and the data is definitely wrong in the table, so I think it is an import issue. New posts with non-ASCII characters in them work fine, and it looks like the data is being correctly stored in the database as unicode in those cases.

* however *

Changing the generic.php to UTF-8 causes the older posts to display correctly. But I'm a little wary of leaving it like that with incorrect data in the database - it feels a bit like we're sweeping the problem under the carpet

I think as long as it's displaying some of the special characters correctly then it's probably ok. The big thing would be to check a post that is showing improperly in the db, and looking at it in the script to see if it's displaying those characters properly. If it is, and you can edit the post as well, without any issues, then you're probably safe.

You could possibly try changing the charset that MySQL uses before the import is done, as I believe it uses latin1 as the default, but if old posts are displaying properly and new posts are ok, then you might not need to go that route.

Things seem to be functioning ok, I'll go with this solution for now. Thanks!