I am having a general intermittent problem where my UBB becomes very very slow at times during the day. Load avg go from below 1 to upwards of 25-30. New posts can take up to 3 mins before they get posted.

I am running UBB 6.7.0. The accelerator and cache are in use. I have be running UBB since 1998 and have never seen my machine become so bogged down.

Do you think this is all load dependent or might I have something not configured properly?

http://www.ibsgroup.org/cgi-local/ubbcgi/ultimatebb.cgi


'ps' and 'top' commands do show an enormous amount of perl activity.

This is the kind of thing I am seeing in 'top' when the slow down occurs:

15:01:50 up 5:03, 1 user, load average: 7.19, 6.37, 5.22
64 processes: 53 sleeping, 11 running, 0 zombie, 0 stopped
CPU states: 95.8% user 4.1% system 0.0% nice 0.0% iowait 0.0% idle
Mem: 512588k av, 412252k used, 100336k free, 0k shrd, 45052k buff
335608k actv, 48400k in_d, 4788k in_c
Swap: 1028148k av, 0k used, 1028148k free 234280k cached

PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND
10097 nobody 23 0 11520 11M 1720 R 11.9 2.2 0:02 0 ultimatebb.cg
10100 nobody 23 0 10080 9.8M 1716 R 11.9 1.9 0:02 0 ultimatebb.cg
10107 nobody 21 0 8952 8952 1712 R 11.9 1.7 0:00 0 ultimatebb.cg
10094 nobody 25 0 17936 17M 1720 R 11.5 3.4 0:04 0 ultimatebb.cg
10103 nobody 22 0 9976 9976 1720 R 9.9 1.9 0:01 0 ultimatebb.cg
10104 nobody 22 0 10336 10M 1724 R 9.9 2.0 0:01 0 ultimatebb.cg
10105 nobody 22 0 10116 9.9M 1724 R 9.9 1.9 0:01 0 ultimatebb.cg
10106 nobody 21 0 8576 8576 1708 R 9.9 1.6 0:00 0 ultimatebb.cg
10108 nobody 21 0 3424 3424 1612 R 4.3 0.6 0:00 0 ultimatebb.cg

A 'ps -ax':

PID TTY STAT TIME COMMAND
1 ? S 0:05 init [3]
2 ? SW 0:00 [migration/0]
3 ? SW 0:00 [keventd]
4 ? SW 0:00 [kapmd]
5 ? SWN 0:00 [ksoftirqd_CPU0]
10 ? SW 0:00 [bdflush]
6 ? SW 0:00 [kswapd]
7 ? SW 0:00 [kscand/DMA]
8 ? SW 0:14 [kscand/Normal]
9 ? SW 0:00 [kscand/HighMem]
11 ? SW 0:00 [kupdated]
12 ? SW 0:00 [mdrecoveryd]
16 ? SW 0:01 [kjournald]
618 ? SW 0:00 [kjournald]
885 ? S 0:00 syslogd -m 0
889 ? S 0:00 klogd -x
926 ? S 0:03 /usr/sbin/named -u named -c /etc/named.conf -u named
940 ? S 0:00 /usr/sbin/sshd
951 ? S 0:00 xinetd -stayalive -pidfile /var/run/xinetd.pid
970 ? S 0:00 sendmail: accepting connections
982 ? S 0:00 /usr/sbin/httpd
991 ? S 0:00 proftpd: (accepting connections)
998 ? S 0:01 /usr/local/jre1.3.1/bin/i386/native_threads/java -Dus
1013 ? S 0:00 crond
1053 tty1 S 0:00 /sbin/mingetty tty1
1054 tty2 S 0:00 /sbin/mingetty tty2
1055 tty3 S 0:00 /sbin/mingetty tty3
1056 tty4 S 0:00 /sbin/mingetty tty4
1057 tty5 S 0:00 /sbin/mingetty tty5
1058 tty6 S 0:00 /sbin/mingetty tty6
8290 ? S 0:02 /usr/sbin/httpd
8784 ? S 0:01 /usr/sbin/httpd
8866 ? S 0:01 /usr/sbin/httpd
8884 ? R 0:02 /usr/sbin/httpd
8885 ? S 0:01 /usr/sbin/httpd
8894 ? S 0:01 /usr/sbin/httpd
8895 ? R 0:01 /usr/sbin/httpd
8906 ? S 0:01 /usr/sbin/httpd
8909 ? S 0:02 /usr/sbin/httpd
8924 ? S 0:01 /usr/sbin/httpd
8927 ? S 0:01 /usr/sbin/httpd
9314 ? S 0:01 /usr/sbin/httpd
9659 ? S 0:00 /usr/sbin/httpd
9661 ? S 0:00 /usr/sbin/httpd
9911 ? S 0:00 /usr/sbin/httpd
9914 ? S 0:00 /usr/sbin/httpd
9915 ? S 0:00 /usr/sbin/httpd
9917 ? S 0:00 /usr/sbin/httpd
9919 ? S 0:00 /usr/sbin/httpd
9927 ? S 0:00 /usr/sbin/httpd
10044 ? S 0:00 /usr/sbin/sshd
10047 ? S 0:00 /usr/sbin/httpd
10048 ? S 0:00 /usr/sbin/sshd
10049 pts/0 S 0:00 -bash
10115 ? S 0:01 /usr/local/bin/perl /home/ibsgroup/public_html/cgi-lo
10127 ? R 0:03 /usr/local/bin/perl /home/ibsgroup/public_html/cgi-lo
10132 ? R 0:00 /usr/local/bin/perl /home/ibsgroup/public_html/cgi-lo
10133 ? R 0:00 /usr/local/bin/perl /home/ibsgroup/public_html/cgi-lo
10134 ? R 0:00 /usr/local/bin/perl /home/ibsgroup/public_html/cgi-lo
10135 pts/0 R 0:00 ps -ax

Some questions:

How many user are active at any given time?
Do you have any code hacks installed?
About how many page views are you getting in a month on your forums?
What is your cache percentage?

I'm sure Charles will have a few more questions/answers than I have, but that's a good start

I average 50-100 users active at once. My max is 240.

No code hacks. All have now been incorporated into the base product. I have only added to the navigation bar in public_common.pl

550,000 page view per month on average

Cache hit is roughly 48%

What are the stats on the server? What type of connection is the server on? Who is the host of the server? Who is on the server becides just yourself (if you know)?

Mainly, when it comes to who is on the server, are you are on a shared or a dedicated?

After a second look, I notice that you have over 23,000 registered users on your boards. If I'm not mistaken, once you get over 20,000 users you risk some performance issues due to the number of directory entries in the Members directory.

It is a dedicated server running RedHat 9.0.

CPU: Intel Celeron 1.7 Ghz
RAM: 512 MB RAM
Hard Drive: 80 GB IDE

Hosting with Interland out of the Dialtone data center in Florida.

10Mb. connection to switch

Virtually one host running on the server. Some single page websites with static ip's.

What you're seeing is "normal"... though that sounds too positive.

Basically, it's a side-effect of the file locking scheme UBB uses. When one instance needs to lock a file, it locks both the file in question and a global file lock. While the global file lock is in place, no other currently running instances of the UBB can perform any file I/O operation... and on a system that uses files for everything, that brings things to a grinding halt if a single UBB process needs to take forever and a day to do something.

That's just about enough server for that board... with high cache hits like that, you shouldn't be putting too much of a dent on that server.

I suggest turning off Recent Visitors, then pruning away about half of your members. Performance starts to drop around the 10,000 mark, and we generally advise not going over 20,000.

Those two steps will both reduce the frequency and duration of the file locks, and also reduce the time it takes to open member profiles, resulting in what will hopefully be a noticable speedup (and thereby also reducing the duration of file locks, which in itself will speed things up more...)

CC, can large archives slow down a UBB? ie 308,000 posts?

Thanks for all the info.

Do you think it is time for me to consider UBB.threads?

Large read-only archives are rarely touched. They would be a performance hit, but not nearly as much as a live, active forum.

You have nearly a half million messages... you could consider UBB.threads, but only if the advise I gave above fails to make performance acceptable. Otherwise, there's no need to move yet. Now, if your traffic were to double, then I'd advise moving.

However, with such a large message base, you would almost certainly need beefier hardware so MySQL can perform well.