[rsyslog] threading - was: output plugin calling interface

[david at lang.hm]

On Wed, 6 May 2009, Rainer Gerhards wrote:

>> -----Original Message-----
>> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
>> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
>>
>> On Wed, 6 May 2009, Rainer Gerhards wrote:
>>
>>>> as I understand it (this is part of the diagram I didn't get created
>>>> yet )-:
>>>
>>> Actually, I don't know how you could envision that other than by drawing
> a
>>> line where a queue exists in the diagrams that exists. Honestly. I have
> no
>>> clue how other I could show this graphically...
>>>
>>>>
>>>> rsyslog basicly breaks down into the following pieces from a
>>>> process/thread point of view
>>>>
>>>
>>> But with words it goes better ;) Note that below is current multi-dequeue
>> git
>>> branch, older versions (especially v3) are somewhat different:
>>
>> at this point I'm going to be lazy and just concentrate on the current
>> version ;-)
>>
>>>
>>> master thread (housekeeping, handling signals, etc)
>>>
>>> N input threads
>>>   recieve messages
>>>   put the message in the main queue
>>>
>>>
>>> N main queue worker threads
>>>   process messages in the main queue
>>>   foreach message
>>>     parse message
>>>     foreach filters
>>>       if filter applies
>>>          put message into the action queue
>>>          [if action queue mode is DIRECT call processAction()]
>>>
>>> N action queue worker threads
>>>   process messages in the action queue
>>>   foreach message
>>>      processAction
>>>
>>> def processAction:
>>>      format the message
>>>      call doAction()
>>>
>>> That's the very rough picture. If you take it together with the message
> flow
>>> diagrams, you'll see that a threading boundary is wherever a queue is.
>>
>> mostly, but not completely (just enough mismatch to cause confusion for
>> me)
>>
>>> Does that help?
>>
>> yes, it helps a _lot_.
>>
>> I think that in most cases where I have said doAction() in the last week,
>> a better fit for what I intended would have been processAction()
>>
>
> We must be very careful here (and I'll check if I wasn't careful enough...).
> processAction() is generic core engine code. doAction() is provided by the
> plugin.

clarifying one point from earlier, this seems to say that the data stored 
in an action queue is exactly the same as the data stored in the main 
queue (specificly, it's the msg structure, not the formatted string)

when processAction() runs, it then does the output formatting and passed 
the formatted message to doAction()

is this correct?

David Lang