[rsyslog] rsyslog - what's next?

[david at lang.hm]

On Mon, 13 Jul 2009, RB wrote:

> On Mon, Jul 13, 2009 at 15:58, <david at lang.hm> wrote:
>> at the time I did quite a number of strace dumps to show how much time was
>> being eaten up in the system calls.
>
> Hopefully some of the tooling Rainer is considering will better enable
> performance monitoring and we can test the effect of some of these
> suggested changes.  I don't completely discount strace as a
> performance testing tool, but it's inherently intrusive, making
> Heisenberg quite happy.

absolutly, especially when you enable timing of the calls !!! (a 
gettimeofday call before and after evey syscall)

but if the program hasn't been examined to try and reduce the number of 
syscalls it makes, strace is a good place to start.

once you get down to a minimum of nessasary syscalls, strace stops being 
useful.

>> well, if you are really wanting audit-grade logging, will you use anything
>> other than the IP address (or what is already in the message)? any lookup
>> that you do is a potential for corruption.
>>
>> I'm fine with the default being to do a lookup for each item. I just want
>> a way to avoid it, and if I can do that with a cache instead of having to
>> disable DNS, I will do so.
>>
>> it's very common for servers to need to disable DNS lookups for their
>> logs. do a quick search for apache dns performance and you will find that
>> it's a very common problem people have there if they enable lookups on the
>> sender's IP address
>
> Indeed; I've generally eschewed DNS for such critical pieces of
> infrastructure, instead opting for resolution during analysis and
> correlation with other logs.  That largely stems from network security
> work and the associated disdain for using DNS entries for rules.  Even
> so, I'm curious whether a properly tuned library-integrated cache like
> nscd (as opposed to a local caching resolver) would provide a
> sufficient performance increase in the interim.

I'll try and get a chance to look into that

> I spent a couple of hours this evening digging through the related
> code with the intent of starting work on delayed resolution, moving
> the lookup to runtime/msg.c:getHOSTNAME.  It might not be the right
> place, but would have the advantage of only resolving hosts you
> explicitly request it for.  Unfortunately, the AllowedSender pragma is
> a sticking point, forcing per-packet resolution; otherwise, it seemed
> a pretty trivial conversion.  I don't know how much (if any)
> performance increase that approach would grant, but it would at least
> move a blocking call out of that single thread.  I may experiment with
> adding a "NoAllowedSenderDNS" later, but it's beyond my time
> flexibility right now.

interesting thought.

David Lang