[rsyslog] untra-reliable speed test

[Rainer Gerhards]

Could you give it a try without the checkpoint interval? That should make a difference.

More answers when i am at a real machine ;)

----- Ursprüngliche Nachricht -----
Von: "david at lang.hm" <david at lang.hm>
An: "rsyslog-users" <rsyslog at lists.adiscon.com>
Gesendet: 12.05.09 23:05
Betreff: Re: [rsyslog] untra-reliable speed test

On Tue, 12 May 2009, Rainer Gerhards wrote:

> Interesting, will look at details tomorrow...

the strace results show a lot of lock contention

> With disk queues, there is always only a single queue worker (the disk queue is purely sequential).

interesting, the OS can do enough in parallel that it may be worth looking 
into this if we ever go in the direction of optimizing this mode.

this was all with rsyslog 4.1.7

David Lang

> rainer
>
> ----- Urspr?ngliche Nachricht -----
> Von: "david at lang.hm" <david at lang.hm>
> An: "rsyslog-users" <rsyslog at lists.adiscon.com>
> Gesendet: 12.05.09 22:23
> Betreff: Re: [rsyslog] untra-reliable speed test
>
> I've completed my first round of testing
>
> this is a a fusionio SSD card with a 8-core opteron system, 8G ram
>
> running debian Lenny (debian 5) 2.6.26 kernel
>
> rsyslog.conf
>
> $ModLoad imuxsock # provides support for local system logging
> $ModLoad imklog   # provides kernel logging support (previously done by rklogd)
> $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
> $WorkDirectory /logs
> $HUPisRestart off
> $MainMsgQueueCheckpointInterval 1
> $MainMsgQueueFilename mainq
> $MainMsgQueueType disk
> $OptimizeForUniprocessor off
>
> #$ActionfileEnableSync on
> #$ActionQueueCheckpointInterval 1
> #$ActionQueueFileName queue1
> #$ActionQueueType disk
> *.*             /logs/messages;RSYSLOG_TraditionalFileFormat
>
>
> input provided by cat largefile | logger
>
>
> I did tests with and without the action queue stuff enabled
>
> the results were not quite what I expected, but interesting
>
> xfs w/ actionqueue   1200/sec
> xfs                  2000/sec
> ext3 w/actionqueue   2000/sec
> ext3                 4600/sec
> ext4 w/actionqueue   2000/sec
> ext4                 4000/sec
> ext2 w/actionqueue   5300/sec
> ext2                 7400/sec
>
> note that with ext2 I don't think the input could keep up (there were not 
> multiple queue files the way there were for all the others), when I 
> shifted to infile as input the ext2 rate increased to ~7800/sec, and the 
> cpu utilization dropped by 50-70%
>
> I have not yet tried anything with multiple worker threads.
>
> I captured some strace files and have posted them at 
> http://rsyslog.lang.hm/rsyslog
>
> David Lang
>
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com
_______________________________________________
rsyslog mailing list
http://lists.adiscon.net/mailman/listinfo/rsyslog
http://www.rsyslog.com