Previous Thread
Next Thread
Print Thread
Hop To
How to deal with a large board without deleting old posts #166018 10/13/2006 6:28 PM
Joined: Aug 2004
Posts: 458
Conrad Offline OP
Addict
OP Offline
Addict
Joined: Aug 2004
Posts: 458
I think everyone who runs a large forum with tons of posts runs into that problem sooner or later: when do I start deleting old stuff to give the server more breathing space?

It's not uncommon to have a posts table that takes up 500 megs, making it by FAR the largest table in the entire database.

Maybe Rick and all of us here can come up with a good way to maintain a large board without having to keep all the old posts in the main database, but also without erasing them completely (I recall that Classic allowed a user to export/archive topics that could be imported back at a later date).

What if instead of pruning topics (which you would still be able to do) you could "archive" topics into another database table (w3t_ArchivePosts.sql)?

These topics would then not burden the main posts table (making the forums run faster) and would only be available via the search function.

Any other ideas on how to tackle this issue?

Re: How to deal with a large board without deleting old posts [Re: Conrad] #166021 10/13/2006 6:41 PM
Joined: Aug 2006
Posts: 1,640
jgeoff Offline
Pooh-Bah
Offline
Pooh-Bah
Joined: Aug 2006
Posts: 1,640
I love the idea of a separate Archive... especially if they could be displayed on the main forum for browsing (as well as searching, which I assume would be a separate search function)



GangsterBB.NET (Ver. 7.6.1.1)
2007 Content Rulez Contest - Hon Mention
UBB.classic 6.7.2 - RIP
Browsers: Chrome, Firefox, & Safari (iPhone); No IE, ever!
Re: How to deal with a large board without deleting old posts [Re: jgeoff] #166031 10/13/2006 7:23 PM
Joined: Jul 2006
Posts: 2,143
David Dreezer Offline
Pooh-Bah
Offline
Pooh-Bah
Joined: Jul 2006
Posts: 2,143
Good idea. Move archives into a separate database table that can grow real large as it's rarely used.


This thread for sale. Click here! [Linked Image from navaho.infopop.cc]
Re: How to deal with a large board without deleting old posts [Re: David Dreezer] #166036 10/13/2006 7:53 PM
Joined: Jun 2006
Posts: 15,844
Gizmo Offline
UBB.threads Developer
Offline
UBB.threads Developer
Joined: Jun 2006
Posts: 15,844
More recent versions of MySQL have an "Archive" table type that would be perfect here (although older versions don't have it, so it'd be a compatability issue).


I am a Web Development Contractor, I do not work for UBBCentral. I have provided free User to User Support since the beginning of these support forums.
Need to Upgrade?
Forums: A Gardeners Forum Scouters World
UBB.threads: UBBWiki, UBB Styles, UBB.Sitemaps
Longtime Supporter & Resident Post-A-Holic
VNC Web Services: Code Modifications, Upgrades, Styling, Coding Services, Disaster Recovery, and more!
Re: How to deal with a large board without deleting old posts [Re: Gizmo] #166089 10/14/2006 7:49 AM
Joined: Aug 2004
Posts: 458
Conrad Offline OP
Addict
OP Offline
Addict
Joined: Aug 2004
Posts: 458
Well Dave, dividing a 500+ meg posts.sql table into two sure sounds better than leaving everything in a single table. Surely you can come up with a more constructive answer than that, perhaps even offer your professional view on what would work best here.


Re: How to deal with a large board without deleting old posts [Re: Conrad] #167014 10/24/2006 6:55 PM
Joined: Aug 2004
Posts: 458
Conrad Offline OP
Addict
OP Offline
Addict
Joined: Aug 2004
Posts: 458
Dave, Gizmo,

How about a different database table for each forum as opposed to one for all posts?

When a board grows, it's the posts table that surpasses the rest in terms of size.

Or what about allowing the admin to create new forums with a tickbox to use a different table for it?

Re: How to deal with a large board without deleting old posts [Re: Conrad] #167094 10/25/2006 5:02 PM
Joined: Jul 2006
Posts: 2,143
David Dreezer Offline
Pooh-Bah
Offline
Pooh-Bah
Joined: Jul 2006
Posts: 2,143
Originally Posted by Conrad
Well Dave, dividing a 500+ meg posts.sql table into two sure sounds better than leaving everything in a single table. Surely you can come up with a more constructive answer than that, perhaps even offer your professional view on what would work best here.



? I'm agreeing that a separate archive table is a good idea. It would get rarely used, wouldn't get stuck in memory, wouldn't get scanned often, would be a good idea.

I don't understand why you responded as you did when I'm agreeing with you, Conrad.


This thread for sale. Click here! [Linked Image from navaho.infopop.cc]
Re: How to deal with a large board without deleting old posts [Re: David Dreezer] #167095 10/25/2006 6:00 PM
Joined: Jun 2006
Posts: 46
Jamin Offline
newbie
Offline
newbie
Joined: Jun 2006
Posts: 46
I think he thought you were being sarcastic.

---Jamin


"Those who would give up essential liberty to purchase a little temporary safety, deserve neither liberty nor safety."
Re: How to deal with a large board without deleting old posts [Re: Jamin] #167097 10/25/2006 6:14 PM
Joined: Aug 2004
Posts: 458
Conrad Offline OP
Addict
OP Offline
Addict
Joined: Aug 2004
Posts: 458
My sincere apologies, Dave.

As Jamin mentioned I wrongly read it as sarcasm.

Just trying to push forth an idea for large-scale UBBT boards as it doesn't seem feasible to leave everything in a single table. Sooner or later with a popular board it will grow to 300mb, 500mb, 700mb, 1GB+. And at some point parts of the forums (probably the oldest threads) are either going to have to be deleted or moved.

There is also another idea: instead of moving threads manually there could be a single CP admin option to decide when certain threads would automatically end up in the archive. This would be global and would apply to all threads. Say if the admin sets this to 2 years - then all posts which haven't had a reply within those 2 years would automatically be moved to the archive.

- people could either search the main site, or if they are looking for something which they think might be older than 2 yrs then they have the option to search the archive. Maybe the search engine can even encompass both altogether in a single search query.

- if things ever get slow or out of hand, a single click could turn the archive off for the time being, until everything is moved to a new server/more memory is installed/etc.

Either way, archived posts would only be available via a special search function so they would not burden the main board by being part of the main database that 99% of users access.

Re: How to deal with a large board without deleting old posts [Re: Conrad] #167100 10/25/2006 6:39 PM
Joined: Jun 2006
Posts: 15,844
Gizmo Offline
UBB.threads Developer
Offline
UBB.threads Developer
Joined: Jun 2006
Posts: 15,844
I always read Dave as being sarcasm, don't know him too well wink...

I do enjoy the idea of the archive though...


I am a Web Development Contractor, I do not work for UBBCentral. I have provided free User to User Support since the beginning of these support forums.
Need to Upgrade?
Forums: A Gardeners Forum Scouters World
UBB.threads: UBBWiki, UBB Styles, UBB.Sitemaps
Longtime Supporter & Resident Post-A-Holic
VNC Web Services: Code Modifications, Upgrades, Styling, Coding Services, Disaster Recovery, and more!
Re: How to deal with a large board without deleting old posts [Re: Gizmo] #167108 10/25/2006 9:15 PM
Joined: Jul 2006
Posts: 2,143
David Dreezer Offline
Pooh-Bah
Offline
Pooh-Bah
Joined: Jul 2006
Posts: 2,143
I'm totally in agreement with the idea of an archive table. If done right the end user has no idea the messages are in a second table, are treated differently programatically, not should they.

Look, in a forum that has 276 pages of posts the average user will look at the first 10, occasionally the last 5, but almost never will they look at pages 15 through 270. So, why keep those in the primary posts table?


This thread for sale. Click here! [Linked Image from navaho.infopop.cc]
Re: How to deal with a large board without deleting old posts [Re: David Dreezer] #167249 10/29/2006 10:59 AM
Joined: Jun 2006
Posts: 3,837
Ian Offline
Carpal Tunnel
Offline
Carpal Tunnel
Joined: Jun 2006
Posts: 3,837
Indeed laugh Looking forward to an archive smile

Re: How to deal with a large board without deleting old posts [Re: Ian] #167331 10/30/2006 6:03 PM
Joined: Jul 2006
Posts: 2,143
David Dreezer Offline
Pooh-Bah
Offline
Pooh-Bah
Joined: Jul 2006
Posts: 2,143
Well, the point of it is, that if you do it right then, to the end user, it should not appear as an archive at all.


This thread for sale. Click here! [Linked Image from navaho.infopop.cc]
Re: How to deal with a large board without deleting old posts [Re: David Dreezer] #167369 10/31/2006 3:00 PM
Joined: Jun 2006
Posts: 3,837
Ian Offline
Carpal Tunnel
Offline
Carpal Tunnel
Joined: Jun 2006
Posts: 3,837
agreed


Forum Search
ShoutChat Box
Comment Guidelines: Do post respectful and insightful comments. Don't flame, hate, spam.
Recent Topics
Unable to save General settings
by amciotola - 10/13/2019 4:23 AM
I see signature line images, but no avatars
by Morgan - 10/12/2019 2:21 PM
Show PDF
by Geoff - 10/09/2019 1:33 AM
Subscription’s Auto Return URL for PayPal
by Geoff - 09/27/2019 12:10 AM
pm's sent directly from view Private messages,
by Ruben - 09/23/2019 3:35 PM
Who's Online Now
2 registered members (JAISP, Ruben), 82 guests, and 369 spiders.
Key: Admin, Global Mod, Mod
Random Gallery Image
Latest Gallery Images
Artistic structures
Artistic structures
by isaac, August 29
Stones
Stones
by isaac, August 19
Amusing Terain Scenics
Amusing Terain Scenics
by isaac, August 19
Sky places
Sky places
by isaac, August 19
Powered by UBB.threads™ PHP Forum Software 7.7.4
(Snapshot build 20190930)