Previous Thread
Next Thread
Print Thread
Hop To
Joined: Jun 2006
Posts: 956
Old Hand
Old Hand
Joined: Jun 2006
Posts: 956
Hi,

today I try out Threads7. I get the beta and the import script. I setup threads with just create the tables and then I run the importer. Seems good up to this point.

The importer begin to work and do something realy fast. As I note he stop a moment while importing the posts (=120) and continue importing the personal mail. I have up to 100.000 posts.

The script finish and I can logon with my just imported admin accounts. All boards are created, users seems to be fine but only a handfull of posts are ready.

Is there any way to get a logfile from the importer and some kind of debug view?

My Threads6 board is old and was years ago imported from classic. I fear that some inconsistency exists in the posting table that confuse the importer. Is some kind of check available to test this? What can I do now?


my board: http://www.dragonclan-forum.de
my hobby: http://www.biker-reise.de
Ich kann bei Fragen zu UBBthreads in Deutsch weiterhelfen oder es zumindest versuchen
Joined: Jun 2006
Posts: 956
Old Hand
Old Hand
Joined: Jun 2006
Posts: 956
I m on the way ... seems that I have very old usernames from imports in my database that contains / [ ] and , in B_AnonName.
Now I edit the db and run the importer again. We will see


my board: http://www.dragonclan-forum.de
my hobby: http://www.biker-reise.de
Ich kann bei Fragen zu UBBthreads in Deutsch weiterhelfen oder es zumindest versuchen
Joined: Jun 2006
Posts: 956
Old Hand
Old Hand
Joined: Jun 2006
Posts: 956
All usernames fixed but this doesn't fix my problem. There must be a post somewhere that give an error, but the import is to fast so I cant see any message. It imports 2803 postings from 100.000

/edit: is there a limit how long a imported thread can be? I have some talk threads with thousends of replys.

/edit2: Seems I found it:

Importing Topic (Frage & Antwort (Endlosthread)) and it's 2227 replies.
Large Topic Encountered. The rest of the topics in this batch will be handled on the next refresh.

Done! Now importing your titles, notes, banlists. (please wait)
(threads_import.php?ubb=extras)

but he dont continue the batch and jumps to:

importing Banned Host: 85.255.117.%
Importing Banned Email: jerk@jerk.com

Last edited by Zarzal; 07/19/2006 9:39 AM.

my board: http://www.dragonclan-forum.de
my hobby: http://www.biker-reise.de
Ich kann bei Fragen zu UBBthreads in Deutsch weiterhelfen oder es zumindest versuchen
Joined: Jun 2006
Posts: 46
J
newbie
newbie
J Offline
Joined: Jun 2006
Posts: 46
Same deal here, except I'm on a much smaller forum. The importer brought in 949 out of 1498 topics, and 7703 out of 12896 posts. It missed a handful of posts from every forum, and left two of the forums empty entirely.

Everything else with the importer seemed to work fine. Got all my users, even got the groups and forum permissions, which surprised me. I was expecting to have to reset all that.

Anyway, yeah, just throwing my $0.02 in here, since Zarzal was reporting the same problem. Otherwise great job!

---Jamin


"Those who would give up essential liberty to purchase a little temporary safety, deserve neither liberty nor safety."
Joined: Jun 2006
Posts: 9,242
Likes: 1
R
Former Developer
Former Developer
R Offline
Joined: Jun 2006
Posts: 9,242
Likes: 1
Jamin, do you have any extremely large posts, one with alot of replies, as well? I think it's a problem when it runs into one of those that I need to fix.

Joined: Jun 2006
Posts: 9,242
Likes: 1
R
Former Developer
Former Developer
R Offline
Joined: Jun 2006
Posts: 9,242
Likes: 1
Thanks for all of the info Zarzal. It's definitely a problem with the large posts as you've found. I have to do some special tricks for those so we don't run into timeout problems and I don't think it's working quite right just yet. Will be working on this more.

Joined: Jun 2006
Posts: 9,242
Likes: 1
R
Former Developer
Former Developer
R Offline
Joined: Jun 2006
Posts: 9,242
Likes: 1
I think I found the problem, just a quick little change. If you edit your threads_import.php file line 773 has a block of code that looks like this:

PHP Code
$end = $end - (20 - $topic_count);
break; 

Change that, to this:

PHP Code
$end = $end - (20 - $topic_count);
$more = true;
break; 


Joined: Jun 2006
Posts: 46
J
newbie
newbie
J Offline
Joined: Jun 2006
Posts: 46
Rick, I have a few posts with 100+ replies, but no really really big ones like Zarzal. My host is currently having a lot of issues with one of their file servers, so everything is slow as nuts right now, but I'll try that fix as soon as things pick up and see if it worked.

---Jamin


"Those who would give up essential liberty to purchase a little temporary safety, deserve neither liberty nor safety."
Joined: Jun 2006
Posts: 9,242
Likes: 1
R
Former Developer
Former Developer
R Offline
Joined: Jun 2006
Posts: 9,242
Likes: 1
Ok, right now a large topic is considered 200+ replies. So if you have any with over 200 it will skip the rest without that fix.

Joined: Jun 2006
Posts: 3,837
I
Ian Offline
Carpal Tunnel
Carpal Tunnel
I Offline
Joined: Jun 2006
Posts: 3,837
I have two with over 10,000 posts each - eek!! Goodness knows how many are over 200 smile

Looks like I will holding fire for a while smile

Joined: Jun 2006
Posts: 956
Old Hand
Old Hand
Joined: Jun 2006
Posts: 956
I do the modification and the import runs right now. Seems good but I have to wait a while for finish smile Thanks.


my board: http://www.dragonclan-forum.de
my hobby: http://www.biker-reise.de
Ich kann bei Fragen zu UBBthreads in Deutsch weiterhelfen oder es zumindest versuchen
Joined: Jun 2006
Posts: 9,242
Likes: 1
R
Former Developer
Former Developer
R Offline
Joined: Jun 2006
Posts: 9,242
Likes: 1
Originally Posted by Ian
I have two with over 10,000 posts each - eek!! Goodness knows how many are over 200 smile

Looks like I will holding fire for a while smile


That should be fine. Basically what it does, is it will only import the one topic and then refresh right after it to avoid a browser timeout. 200 is just a number I pulled out of my...um....head, and can probably be much higher. wink

Joined: Jun 2006
Posts: 3,837
I
Ian Offline
Carpal Tunnel
Carpal Tunnel
I Offline
Joined: Jun 2006
Posts: 3,837
so my 10,000 post threads should be fine (I think one is around 12,500 posts long at the moment)

Thanks.

Joined: Jun 2006
Posts: 9,242
Likes: 1
R
Former Developer
Former Developer
R Offline
Joined: Jun 2006
Posts: 9,242
Likes: 1
It should be. The SQL part isn't really the issue, the queries are very quick, the only problem I've had to work on avoiding is a browser timeout. Only 1 way to find out for sure though is by setting up a test forum for a test run wink Yours would be a great experiment laugh

Joined: Jun 2006
Posts: 3,837
I
Ian Offline
Carpal Tunnel
Carpal Tunnel
I Offline
Joined: Jun 2006
Posts: 3,837
My only issue at the moment is upgrading to 6.5.4 - I run 6.5.2 and cannot upgrade until I actually switch.

My only option is to install a duplicate site, and upgrade that.

I am assuming that one can't just set up a beta and then import a 6.5.2 database.

Joined: Jun 2006
Posts: 9,242
Likes: 1
R
Former Developer
Former Developer
R Offline
Joined: Jun 2006
Posts: 9,242
Likes: 1
The import will actually work as long as you are at least running version 6.5. So since you are running 6.5.2 you can do just that. Setup the beta and then do the import. wink

Joined: Jun 2006
Posts: 3,837
I
Ian Offline
Carpal Tunnel
Carpal Tunnel
I Offline
Joined: Jun 2006
Posts: 3,837
Ok cool smile Thanks.


Joined: Dec 2003
Posts: 1,796
Pooh-Bah
Pooh-Bah
Joined: Dec 2003
Posts: 1,796
I've found through many many imports that you'll probably have better results with the timeout issues if you use firefox. With the newest changes IE will probably be ok, but firefox doesn't time out as badly.


- Allen
- ThreadsDev | PraiseCafe
Joined: Jun 2006
Posts: 46
J
newbie
newbie
J Offline
Joined: Jun 2006
Posts: 46
Rick, just FYI the fix worked great, and it imported everything correctly this time.

---Jamin


"Those who would give up essential liberty to purchase a little temporary safety, deserve neither liberty nor safety."
Joined: Jun 2006
Posts: 9,242
Likes: 1
R
Former Developer
Former Developer
R Offline
Joined: Jun 2006
Posts: 9,242
Likes: 1
Thanks for the update Jamin. Good to hear!

Joined: Jun 2006
Posts: 956
Old Hand
Old Hand
Joined: Jun 2006
Posts: 956
my import is finish and it works. More then 100.000 potings in nearly 9.000 topics and 11.000 private topics imported. All fine.

the only thing I note (and it seems to be by design): all polls start from 0, the old results get not transferd.


my board: http://www.dragonclan-forum.de
my hobby: http://www.biker-reise.de
Ich kann bei Fragen zu UBBthreads in Deutsch weiterhelfen oder es zumindest versuchen
Joined: Jun 2006
Posts: 9,242
Likes: 1
R
Former Developer
Former Developer
R Offline
Joined: Jun 2006
Posts: 9,242
Likes: 1
Hmm, it is supposed to transfer the results. It's ok that they start from 0, but results should have been brought over as well. I'll need to debug this to get it fixed.

Joined: Jun 2006
Posts: 464
Likes: 1
M
Addict
Addict
M Offline
Joined: Jun 2006
Posts: 464
Likes: 1
Originally Posted by Rick
I think I found the problem, just a quick little change. If you edit your threads_import.php file line 773 has a block of code that looks like this:

PHP Code
$end = $end - (20 - $topic_count);
break; 

Change that, to this:

PHP Code
$end = $end - (20 - $topic_count);
$more = true;
break; 

This worked perfect for me today.. My conversion stopped at like 800 posts.. this fixed it.


Happy Customer !!!
Joined: Jun 2006
Posts: 956
Old Hand
Old Hand
Joined: Jun 2006
Posts: 956
Just a note:
Importet postings with included quotes looks different. I guess its related to the diffent styles from ubb6.5 and ubb7 and the handling of quotes. Postings will transfered and not modifed in its body so the new quote layout is not applied.


my board: http://www.dragonclan-forum.de
my hobby: http://www.biker-reise.de
Ich kann bei Fragen zu UBBthreads in Deutsch weiterhelfen oder es zumindest versuchen

Link Copied to Clipboard
ShoutChat
Comment Guidelines: Do post respectful and insightful comments. Don't flame, hate, spam.
Recent Topics
spam issues
by ECNet - 03/19/2024 11:45 PM
Looking for a forum
by azr - 03/15/2024 11:26 PM
Editing Links in Post
by Outdoorking - 03/15/2024 9:31 AM
Question on barkrowler and the like
by Mors - 02/29/2024 6:51 PM
Member Permissions Help
by domspeak - 02/27/2024 6:31 PM
Who's Online Now
1 members (Ruben), 476 guests, and 111 robots.
Key: Admin, Global Mod, Mod
Random Gallery Image
Latest Gallery Images
Los Angeles
Los Angeles
by isaac, August 6
3D Creations
3D Creations
by JAISP, December 30
Artistic structures
Artistic structures
by isaac, August 29
Stones
Stones
by isaac, August 19
Powered by UBB.threads™ PHP Forum Software 8.0.0
(Preview build 20230217)