[rsyslog] rsyslog - what's next?

[RB]

On Mon, Jul 13, 2009 at 15:58, <david at lang.hm> wrote:
> at the time I did quite a number of strace dumps to show how much time was
> being eaten up in the system calls.

Hopefully some of the tooling Rainer is considering will better enable
performance monitoring and we can test the effect of some of these
suggested changes.  I don't completely discount strace as a
performance testing tool, but it's inherently intrusive, making
Heisenberg quite happy.

> well, if you are really wanting audit-grade logging, will you use anything
> other than the IP address (or what is already in the message)? any lookup
> that you do is a potential for corruption.
>
> I'm fine with the default being to do a lookup for each item. I just want
> a way to avoid it, and if I can do that with a cache instead of having to
> disable DNS, I will do so.
>
> it's very common for servers to need to disable DNS lookups for their
> logs. do a quick search for apache dns performance and you will find that
> it's a very common problem people have there if they enable lookups on the
> sender's IP address

Indeed; I've generally eschewed DNS for such critical pieces of
infrastructure, instead opting for resolution during analysis and
correlation with other logs.  That largely stems from network security
work and the associated disdain for using DNS entries for rules.  Even
so, I'm curious whether a properly tuned library-integrated cache like
nscd (as opposed to a local caching resolver) would provide a
sufficient performance increase in the interim.

I spent a couple of hours this evening digging through the related
code with the intent of starting work on delayed resolution, moving
the lookup to runtime/msg.c:getHOSTNAME.  It might not be the right
place, but would have the advantage of only resolving hosts you
explicitly request it for.  Unfortunately, the AllowedSender pragma is
a sticking point, forcing per-packet resolution; otherwise, it seemed
a pretty trivial conversion.  I don't know how much (if any)
performance increase that approach would grant, but it would at least
move a blocking call out of that single thread.  I may experiment with
adding a "NoAllowedSenderDNS" later, but it's beyond my time
flexibility right now.