[rsyslog] untra-reliable speed test

[Rainer Gerhards]

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Friday, May 08, 2009 2:05 AM
> To: rsyslog-users
> Subject: [rsyslog] untra-reliable speed test
> 
> I have a box put togeaterh for a first cut at a speed test of rsyslog in
> untra-reliable mode. the outline below is intended to minimize the number
> of variables.
> 
> the box is a dual quad-core opteron with 8G of ram, one SATA drive and a
> fusionIO SSD PCIE drive, currently running RHEL 5.3 kernel 2.6.18-53
> (redhat stock kernel) I intend to format the SSD with ext2 (as the
> application is providing data integrity, and to avoid the known
> performance problems with ext3 and fsync)

Just a question, because I do not know enough about ext2: does ext2 guarantee
that when an application does fsync, all data, INCLUDING related file system
control structures are written to disk? Or, to phrase it the other way
around, can ext2 guarantee that fsync'ed data can always be read after a
power failure. I think along the lines of some control structures not being
written, thus the fsynced app data may be present on the disk, but cannot be
accessed any longer. In the worst case, would it be possible that a whole
file be lost during a file system check after reboot? 

My *uneducated* understanding is that ext3 does guard against this (thus the
performance problems) but ext2 does not.

If my understanding would be correct (and I don't say so), we would need to
use ext3.
 
> for the rsyslog test I am thinking the following
> 
> useing rsyslog 4.1.7
> enable input file

Not sure if I got this bullet point right. Do you mean you intend to use
imfile for input generation?

In any case, I would suggest to do a test with UDP and one with TCP senders,
both sending at maximum rate. With UDP, we would see a message loss rate,
while with TCP we would see the actual number of messages that the system can
process. So TCP is probably the more meaningful number, but packet loss rate
for UDP - a common use case - would also be interesting, at least I think so.
 
> set the main queue mode to disk
> enable fsyncs everwhere


Just as a reminder: this includes $MainMsgQueueCheckpointInterval 1 (which is
a *real* performance eater and puts a lot of burden on the consistency of the
file system's control structures, thus my question on ext2 vs. ext3 above).

> set the output to log *.* to a file
> 
> run a cron job that rolls the log file once a min and sends a HUP to
> rsyslog
> 
> create a large file of log information
> 
> run this for a while and then count the number of logs in each rolled log
> file. hopefully the number will be reasonably consistant.
> 
> does this sound like a reasonable approach? or is this going to not be
> representitive for some reason?

With the few comments above, I think this is a very reasonable approach and
should provide very good insight.

Actually, I hope that it can prove my point that this setup is too slow
wrong...

Rainer
> 
> David Lang
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com