Is this a server or client problem? Could it be hardware related? There is nothing in my /var/log/messages or syslog to indicate a hardware problem. The UBB appears to be functioning fine.

[Wed Mar 20 10:58:42 2002] [error] [client 216.XX.XX.XX] File does not exist: /home/ibsgroup/public_html/ubb/Forum1/HTML/007357-2.html
syntax error at /home/ibsgroup/public_html/cgi-local/ubbcgi/Modules/vars.pm line 2, near "xí@xí@rror at "
BEGIN failed--compilation aborted at /usr/lib/perl5/5.00503/lib.pm line 3.
BEGIN failed--compilation aborted at /home/ibsgroup/public_html/cgi-local/ubbcgi/ultimatebb.cgi line 45.
[Wed Mar 20 11:10:13 2002] [error] [client 63.XX.XX.XX] Premature end of script headers: /home/ibsgroup/public_html/cgi-local/ubbcgi/ultimatebb.cgi
Unrecognized character 200 at Modules/AutoLoader.pm line 1.
BEGIN failed--compilation aborted at /home/ibsgroup/public_html/cgi-local/ubbcgi/ultimatebb.cgi line 48.
[Wed Mar 20 11:10:19 2002] [error] [client 63.XX.XX.XX] Premature end of script headers: /home/ibsgroup/public_html/cgi-local/ubbcgi/ultimatebb.cgi
Unrecognized character 01 at /usr/lib/perl5/5.00503/strict.pm line 1.
BEGIN failed--compilation aborted at /home/ibsgroup/public_html/cgi-local/ubbcgi/ultimatebb.cgi line 47.
[Wed Mar 20 11:10:27 2002] [error] [client 66.XX.XX.XX] Premature end of script headers: /home/ibsgroup/public_html/cgi-local/ubbcgi/ultimatebb.cgi
Semicolon seems to be missing at /usr/lib/perl5/5.00503/Carp.pm line 4.
Semicolon seems to be missing at /usr/lib/perl5/5.00503/Carp.pm line 7.
Semicolon seems to be missing at /usr/lib/perl5/5.00503/Carp.pm line 9.
String found where operator expected at /usr/lib/perl5/5.00503/Carp.pm line 15, near "croak "We're outta here!""
(Do you need to predeclare croak?)
syntax error at /usr/lib/perl5/5.00503/Carp.pm line 3, near "of caller"
"use" not allowed in expression at /usr/lib/perl5/5.00503/Carp.pm line 14, at end of line
BEGIN not safe after errors--compilation aborted at /usr/lib/perl5/5.00503/Carp.pm line 17.
BEGIN failed--compilation aborted at Modules/UBBCGI/Carp.pm line 5.
BEGIN failed--compilation aborted at /home/ibsgroup/public_html/cgi-local/ubbcgi/ultimatebb.cgi line 50.
[Wed Mar 20 11:10:30 2002] [error] [client 63.XX.XX.XX] Premature end of script headers: /home/ibsgroup/public_html/cgi-local/ubbcgi/ultimatebb.cgi
syntax error at /usr/lib/perl5/5.00503/Exporter.pm line 1, near ">"
BEGIN failed--compilation aborted at /usr/lib/perl5/5.00503/i386-linux/Config.pm line 2.
BEGIN failed--compilation aborted at /usr/lib/perl5/5.00503/lib.pm line 4.
BEGIN failed--compilation aborted at /home/ibsgroup/public_html/cgi-local/ubbcgi/ultimatebb.cgi line 45.
[Wed Mar 20 11:10:41 2002] [error] [client 207.XX.XX.XX] Premature end of script headers: /home/ibsgroup/public_html/cgi-local/ubbcgi/ultimatebb.cgi
[Wed Mar 20 11:10:52 2002] ultimatebb.cgi: /home/ibsgroup/public_html/cgi-local/ubbcgi/ubb_lib_time.cgi: [Wed Mar 20 11:10:52 2002] ultimatebb.cgi: Unrecognized character 01 at /home/ibsgroup/public_html/cgi-local/ubbcgi/ubb_lib_time.cgi line 1.

Mmmmm... if I didn't know better, I'd say you were running NT with the ISAPI DLL, but clearly you're on Unix.

How frequently do errors such as this occur?

--
Charles Capps
Programmer, Infopop Corporation
Please do not contact me privately for support - post on the board or open a support ticket instead!

Yup, RedHat 6.2

A simple grep in the error log shows 96 occurences of "compilation aborted" since March 1. It seems that is printed twice per error so I'd guess 48 times this month.

Can you do me a favor?

Grab as many of those errors as possible, stick them in a text file, and mail it to charles@infopop.com, including the URL to this topic.. I'd like to look at all of the errors.

--
Charles Capps
Programmer, Infopop Corporation
Please do not contact me privately for support - post on the board or open a support ticket instead!

Done.

Thanks - I'll be trudging through those this afternoon.

--
Charles Capps
Programmer, Infopop Corporation
Please do not contact me privately for support - post on the board or open a support ticket instead!

OKay... something is very wrong here. I can't determine what would cause those errors...

--
Charles Capps
Programmer, Infopop Corporation
Please do not contact me privately for support - post on the board or open a support ticket instead!

Anything I can do to help debug?

Do you believe this is a UBB thing?

I'm not really sure what to do.

All of the errors don't exist - if the modules being referenced had the problems that the log says they're having, perl would flat out NOT operate on your server.

We've only seen one thing like this - the ISAPI DLL of ActiveState Perl throws errors EXACTLY like this......

Are you using FastCGI / SpeedyCGI or any similar tool that "speeds up" CGI processes?

--
Charles Capps
Programmer, Infopop Corporation
Please do not contact me privately for support - post on the board or open a support ticket instead!

No, nothing fancy on the box.

I'm running pretty vanilla Redhat 6.2 with these RPMS:

perl-5.00503-10
mod_perl-1.21-10
MySQL-shared-3.23.40-1
php-4.0.5-2.0.0
db3-3.1.17-4.6x
mod_php-4.0.5-2.0.0
mod_gzip-1.3.19.1a-1

I'm going to check my February and January logs to see when this started happening. My hunch is that it started recently because of the increased load caused by Recent Visitors. Just a hunch.

... are you running UBB under mod_perl?

--
Charles Capps
Programmer, Infopop Corporation
Please do not contact me privately for support - post on the board or open a support ticket instead!

Hmm.... that would be affirmative.

AddModule mod_perl.c is in httpd.conf

Re-started httpd to be certain:

[Mon Mar 25 13:51:41 2002] [notice] Apache/1.3.12 (Unix) (Red Hat/Linux) PHP/4.0.5 mod_perl/1.21 mod_gzip/1.3.19.1a configured -- resuming normal operations

Are you onto something?

Well, just because you have mod_perl compiled in, that doesn't automagically make UBB run under it... thank goodness.

Does your UBB randomly forward people to other pages, duplicate output (i.e. one page's contents appended to the bottom of another) or do anything else freaky and otherwise odd?

--
Charles Capps
Programmer, Infopop Corporation
Please do not contact me privately for support - post on the board or open a support ticket instead!

None of the above.

It performs normally other than the occasional burp and those error messages.

When you mentioned mod_perl I remembered reading that UBB doesn't support that, but I wasn't sure how to tell UBB to use or not use mod_perl's features.

I'm starting to believe this is a resource issue rather than anything to do with UBB. I have looked back in my error logs and I have seen these error messages well before I upgraded to UBB 6.X, which generally increased the system load.

I have adjusted the Apache conf variables:

RLimitCPU and RLimitMEM to "max max" to see if this resolves the issue.

I'll keep you posted.

The above conf changes had no effect.

I added a cron job to do a 'ps -fl' and 'uptime' every minute. I seem to have a correlation with a large number of sys_flock calls being made and a larger load and these errors occuring.

Does this ring any bells for you as to being a UBB issue?

000 R nobody 13153 12227 29 70 0 - 10800 - 00:02 ? 00:00:05
perl /home/ibsgroup/public_html/cgi-local/ubbcgi/ultimatebb.cgi
000 S nobody 13154 13115 5 60 0 - 1267 flock_ 00:02 ? 00:00:00
perl /home/ibsgroup/public_html/cgi-local/ubbcgi/ultimatebb.cgi
000 S nobody 13155 12230 10 63 0 - 1267 flock_ 00:02 ? 00:00:00
perl /home/ibsgroup/public_html/cgi-local/ubbcgi/ultimatebb.cgi
000 S nobody 13156 13117 9 63 0 - 1267 flock_ 00:02 ? 00:00:00
perl /home/ibsgroup/public_html/cgi-local/ubbcgi/ultimatebb.cgi
000 S nobody 13157 12215 12 68 0 - 1268 flock_ 00:02 ? 00:00:00
perl /home/ibsgroup/public_html/cgi-local/ubbcgi/ultimatebb.cgi
000 S nobody 13158 13116 16 68 0 - 1267 flock_ 00:02 ? 00:00:00
perl /home/ibsgroup/public_html/cgi-local/ubbcgi/ultimatebb.cgi
000 R nobody 13159 13119 21 71 0 - 1111 - 00:03 ? 00:00:00
perl /home/ibsgroup/public_html/cgi-local/ubbcgi/ultimatebb.cgi

All those flock calls waiting are the UBBs stacking up waiting for the central lockfile to become available. This is actually the designed behavior.

Usually errors like these would correspond to 500 ISEs being thrown all over the place.... have you seen any at all?

--
Charles Capps
Programmer, Infopop Corporation
Please do not contact me privately for support - post on the board or open a support ticket instead!

I'm not sure I can differentiate the errors within the logfile. The errors that users are sending me are no doubt 500 errors.

Lots of "File not found" in the logfile. What would you look for for 500 errors?

Something else to ponder...

I disabled the UBB Accelerator yesterday afternoon to attempt to narrow this down. Since then I have not seen a single error in the error log file.

However, maybe a red herring because the errors appeared before I upgraded to UBB 6.x as evident from my error log files from 6/2001, etc..

[This message was edited by ibsgroup on 29 Mar 02 at 07:01 AM.]

If the errors occured before going to 6.1 (which is when many issues like this made themselves known), then chances are it's something with your local setup that is going awry than something that the UBB is directly causing.

The file not founds within the logs are just that - clients requesting files that aren't there...

--
Charles Capps
Programmer, Infopop Corporation
Please do not contact me privately for support - post on the board or open a support ticket instead!

Yes, I agree. The additional load of 6.1 just made the issue that more noticeable.

Will keep you posted if I ever track this down. Thankfully nothing is getting corrupted.

I believe I have finally tracked down the problem. The search function is causing the problem. The search index's are very very large, ie: up to 30Mb. for each forum!

For the time being I have turned off the search function until I figure out how to resolve this.

FYI, to troubleshoot this I used the Apache mod_status module and captured server-status output, with ExtendedStatus On, in a file through a cron script with the text based web browser 'lynx'. After analyzing the captured data around the time of the error messages I noticed that 3 searches were running at once. I tried this myself by running searches in 3 separate browser sessions on the largest index file and voila, I reproduced the problem.

This is the cron entry for anyone interested:

0-59 * * * * (TERM=vt100;export TERM;lynx -dump http://www.ibsgroup.org/server-status >> /tmp/status.log)

Any ideas how to resolve this other than pruning posts? I read that you've changed how indexing works in 6.2.2. Is this now scraped because 6.3 is on the way?

[This message was edited by ibsgroup on 03 Apr 02 at 09:27 AM.]

Search is rewritten and fixed in 6.3--uses much less memory...it's in beta now. Check out UBBDesign.com.



beezoo.com

Good to hear and thanks.