[rsyslog] output plugin calling interface

[david at lang.hm]

On Fri, 1 May 2009, Rainer Gerhards wrote:

>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com
>> [mailto:rsyslog-bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>>
>> On Fri, 1 May 2009, Rainer Gerhards wrote:
>>
>>> Please let me know if you also find a math model useful
>> (but I'll probably
>>> need to do it in any case, because it helps me clean up my mind...).
>>
>> I think it will help clarify things a lot. with a good model
>> we won't have
>> misunderstandings about what we are talking about.
>
> Yes - and I also think that with the model some complexities disappear. I
> think (hope I am right) the solution will become obvious. I know I am
> investing a lot of time in a tiny portion of the code, but this is one of the
> core elements involving many complexities.
>
>> with my 'binary search' approach, handling permanently bad
>> messages could
>> be as simple as 'too many retries once we hit a batch size of
>> 1' (with a
>> possible option of the output module reporting back that it dectected
>> something that makes retries useless, but this is just an
>> optimization)
>
> Yes, indeed. One quick thought: I see a batch as a set of (msg, state)
> ordered pairs. Once we have procssed it in one action (all of them have
> entered one permanent state), we can than build a subset that we use as the
> new (remaining) batch in the backup actions. So the "bad record search" is
> "just" one facet of many that we need to handle with little and hopefully
> simple code (doing it with 2000 LoC would be rather easy ;)).

I agree with the definition of a batch. Let's see what different states 
you are thinking of.

I am currently assuming that the messages stay in the queue (with the 
state attached) so that if rsyslog restarts (assuming disk queues), it 
will realize that the message hasn't been delivered and try again.

David Lang