[rsyslog] output plugin calling interface

[Rainer Gerhards]

David,

OK, I have uploaded a new version, this time with pseudocode for the batch
processing layer. I have to say I am very optimistic that this is the right
route, differentiating between two different cases of action failure causes
really helped. I think this is something that I could implement right away
and it looks so simple that it seems hard to make a mistake in doing so.

... but I guess you'll find something I have overlooked ;)

This version should be safe to work with, I'll probably not update it any
more today.

Rainer

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Thursday, May 07, 2009 4:13 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] output plugin calling interface
> 
> I've just done another update tot he paper with some pseudocode (hopefully)
> upcoming later today. Pseudocode is not essential.
> 
> > -----Original Message-----
> > From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> > bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> > Sent: Thursday, May 07, 2009 3:54 PM
> > To: rsyslog-users
> > Subject: Re: [rsyslog] output plugin calling interface
> >
> > David,
> >
> > I have now looked at the code and modified it, so I get some "feeling" of
> how
> > it looks and works (it doesn't matter if I need to dump or modify that
> code,
> > it took a few hours, less than writing other things ;)). So I am still
> "only"
> > biased, but not without alternative. The code looks much cleaner than
what
> is
> > in v3, btw.
> >
> > But I think I have also come closer to where our opinions differ. I
> mentioned
> > this morning that we have different design approaches. But that's not the
> > full picture... let me quote you:
> >
> > > > Take care of the new state diagram and be sure to understand that it
> > models
> > > > an own *action state*, not a batch transaction state (that's
different
> > and
> > > > for tomorrow ;)).
> > >
> > > I'm not sure that there can be an error from the inTx stage that would
be
> > > worth retrying. errors there would not be related to outputting the
> > > message, but simply to processing it and preparing it to be output
later.
> > >
> > > in fact, I'm not sure that retry belongs in the message state at all. I
> > > could see it argued that the commit may result in a temporary error
that
> > > could be retried, but is that really something that the action (i.e.
> > > output module) should deal with? or should this be done at the
> transaction
> > > state?
> > >
> > > in reading the page after the diagram, it appears that you are thinking
> > > the same thing, in which case the retry and suspend nodes should be
> > > removed from the state diagram (or there may need to be a suspend node
if
> > > you want the higher levels to be able to try again and the module to
> > > reject it)
> > >
> > >
> > > looking at your pseudocode, I started to re-write it, and I think
things
> > > can be much simpler.
> > >
> > > if the retrys are done above this level, then the only thing that we
need
> > > to do is to not hammer the destination.
> > >
> > > except for the fact that doAction() can trigger an EndTransaction()
> > > internally, there is no reason why doAction() can't take place while
> > > suspended (the output module can be preparing the stuff to send out).
the
> > > only place that needs to deal with the issue is that the
EndTransaction()
> > > should sleep if the state is not itx
> > >
> > > if doAction does beginTransaction() any time it's not in a transaction,
> > > there is no reason to have it as a seperate call.
> > >
> > > so without retries or beginTransaction, is there any reason for
> > > prepareAction() to exist?
> > >
> > > you also will need to detect that the doAction() did endTransaction()
and
> > > that you don't need to issue a endTransaction() for this output module
> now
> > > (until you do the next doAction() )
> >
> > I think we probably have different failure cases on our mind. We touched
> > this, but probably did not make the issue clear enough. I now think that
> > these different classes of failures require different handling, probably
at
> > different layers of the engine. Maybe this can help to combine our both
> > views.
> >
> > I was first tempted to start the description right here in mail, but
> instead
> > I have added some text to the "internals document", hoping that the
> > information may be useful in the future, too (and knowing that I need to
> edit
> > it soon ;)).
> >
> > Note that I have NOT yet updated any other part of the document. It's
> > probably also affected by thinking about failure cases.
> >
> > So, I'd appreciate if you could have a look at sections 3.2 and 3.3 of
> >
> > http://www.rsyslog.com/download/design.pdf
> >
> > Thanks,
> > Rainer
> > _______________________________________________
> > rsyslog mailing list
> > http://lists.adiscon.net/mailman/listinfo/rsyslog
> > http://www.rsyslog.com
> _______________________________________________
> rsyslog mailing list
> http://lists.adiscon.net/mailman/listinfo/rsyslog
> http://www.rsyslog.com