[rsyslog-notify] Forum Thread: Re: imtcp maxsessions - (Mode 'reply')

[noreply at adiscon.com]

User: dlang 
Forumlink: http://kb.monitorware.com/viewtopic.php?p=25683#p25683

Message: 
----------
a couple minor comments

with current versions, you don't need to be as creative with the
facility/severity matches.

For example, I find

if $programname startswith 'rsyslogd-pstats' then /var/log/syslog-stats

I do almost all my filtering based on programname and similar rather than
facility type. There just aren't enough facilities for the filtering that I
want to do, and getting all the senders to use the right one is a
maintenance headache. I completely ignore facility.

much clearer (and less subject to false positive matches) than

 syslog.=debug /var/log/rsyslog-stats

there are several places where you have - in your config (i.e.  mail.*
-/var/log/maillog) with traditional syslog, this meant that syslog could
buffer the message instead of delivering it synchronously. With rsyslog
everything is async, so this is a no-op

you don't need to do *.* for anything

so instead of 

*.* {
action(type="omprog"
binary="path to compiled python binary omitted"
template="RSYSLOG_TraditionalFileFormat")
}

you can just do

action(type="omprog"
binary="path to compiled python binary omitted"
template="RSYSLOG_TraditionalFileFormat")

one thing that may speed up your python programs is that instead of using
the default template, cusomize a template to pass the program only the data
that it needs.

Overall this looks pretty efficient. I'd be interested to see where CPU is
being used (run top and hit H to show the per-thread info)

I would experiment with a queue for the omprog actions, just to make sure
that the python binaries are keeping up with the input. I expect that there
would be no improvement (and possibly a regression) on the CPU used by
rsyslog, but it's worth checking

using global() to set the batch size to some larger number may help
(although this may depend on the buffering between rsyslog on the python
binaries) but overall, this looks pretty efficient. moving from imtcp to
imptcp should help. But I think the key question is why your clients are
disconnecting so rapidly. They should connect and keep the connection open
for a long time (days, weeks) continuing to push data over that one
connection. If they are doing "connect, send message, disconnect, repeat
for the next message" then you would really be better of switching to UDP,
you aren't actually getting much reliability from using TCP, and it is
costing you a lot of overhead.