[rsyslog] High CPU utlization, and memory usage

[david at lang.hm]

On Thu, 9 Jul 2009, Peter Doherty wrote:

> Hi, I recently deployed rsyslog on a server.  I'm using a central
> server that receives logs from currently just 4 other machines.  It's
> using TLS.  It's a pretty basic setup, I didn't do anything fancy.
>
> Twice this week the central server has had issues, and when I logged
> in to check, I saw that rsyslog was using all the swap (2GB) and using
> over 300MB of the 512MB RAM, plus using 70% or more CPU time.

one thing when examining the load is to keep in mind that rsyslog uses 
multiple threads, in linux hit 'H' in top to have it show the individual 
threads.

it's very possible that the high cpu load is due to the encryption 
overhead

one thing that I do when testing is that once I identify which thread is 
eating a significant amount of cpu I do 'strace -p <pid>' for that thread 
for a few seconds, looking at that output I can ususally make a fair guess 
at what the thread is busy working on.

using that much ram would leave me guessing that the ability to write the 
log file stopped, and the queue is filling up.

David Lang

> I'm running version 4.2.0
> Can you give me some idea where to start looking for what's causing
> this?  It seems to run fine for a day or so, and then over a very
> short amount of time, maybe 2-60 minutes, the memory and  cpu usage
> spikes.
>
> Here's a snippet from the /etc/rsyslog.conf if that helps.
>
> Thanks.
> --Peter
>
>
> $ModLoad immark
> $ModLoad imklog
> $ModLoad imuxsock # local messages
> $ModLoad imtcp # TCP listener
>
> $MarkMessagePeriod 1200
>
> # make gtls driver the default
> $DefaultNetstreamDriver gtls
>
> # certificate files
> $DefaultNetstreamDriverCAFile /usr/share/tls/ca.pem
> $DefaultNetstreamDriverCertFile /usr/share/tls/server-cert.pem
> $DefaultNetstreamDriverKeyFile /usr/share/tls/server-key.pem
>
>
> $InputTCPServerStreamDriverAuthMode x509/name
> $InputTCPServerStreamDriverMode 1 # run driver in TLS-only mode
> $InputTCPServerRun 10514 # start up listener at port 10514
>
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
>