[rsyslog] audit-grade queue / batching

[Rainer Gerhards]

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Friday, May 15, 2009 9:32 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] audit-grade queue / batching
> 
> On Fri, 15 May 2009, Rainer Gerhards wrote:
> 
> >> -----Original Message-----
> >> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> >> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> >
> >> one final thought, if a message gets a permanent failure and backup
> >> processing kicks in, can this still be audit-grade?
> >
> > I doubt, but it probably depends on your legislation (or other policies).
> 
> what I was meaning with this questions is if shifting from the primary
> delivery to the backup processing, would this still be the
> 'ultra-reliable' mode where if the system dies before the log message gets
> to the backup destination it will not be lost, but instead retried?

I misunderstood your question. Now thought a bit about the scenarios. You are
right, this does not work with message-permanent failures. This case
previously never existed and so - in theory - any message that was enqueued
could also be processed, even if potentially much later. Now we have messages
that are enqueued and thus taken off the previous queue AND action processing
but may still fail.

I see two approaches at handling this:

a) we enable an action to configure a backup file that shall receive all
message permanent failures. This is simple (not only to implement but to
configure and understand)

b) we push the failed message back to the main queue, but with an indication
that it failed in an action. This is harder to implement and most importantly
harder to understand/configure, but more flexible

Do you see any other possibilities? Which one would you prefer?

Rainer