[rsyslog] multi-message handling and databases
rgerhards at hq.adiscon.com
Tue Apr 21 07:41:48 CEST 2009
> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of david at lang.hm
> Sent: Tuesday, April 21, 2009 3:59 AM
> To: rsyslog-users
> Subject: Re: [rsyslog] multi-message handling and databases
> On Mon, 20 Apr 2009, Rainer Gerhards wrote:
> >>> ... especially if you think about the data flow. At this point, it
> >> may make
> >>> sense to review the data flow. I have described it here:
> >>> http://www.rsyslog.com/Article350.phtml
> one problem with this diagram (and therefor the explination that goes
> it) it that it confuses logical steps with objects with threads.
It was not designed as an in-depth threading description. It is about the
data flow, not which thread does what. Looks like I need to create such a
thing. But you can easily do this. The queue is the threading boundary.
Everything on the left side is one thread, everything on the right site is
another thread (except if the queue runs in direct mode, in which case it is
the same thread to the left and the right).
> I would suggest redrawing the diagram with boxes to delinate thread
> boundries (if I understand this correctly, this would mean splitting
> pre-processing box up into three, each of which would be in the same
> 'thread' box as the input module)
Ok... I see where you are coming from. But is this split really useful? Each
input runs on its own thread, so if we have n input, we need to split the
Preprocessor box into n parts...
> It is also not as clear as it could be what happens when you don't
> action queues. I would suggest a sepeate diagram just showing that
> situation (if I understand it correctly, you could show the single-
> operation, and then have one of the 'actions' be to queue the item into
> action queue for async processing.
This is the core misunderstanding. You ALWAYS have action queues! You can not
"not define action queues". What you can do is not define the operation mode
of the action queue, in which case the default operation mode is used. That
happens to be the non-queueing direct mode. But that doesn't at all change
the picture. Eexcept, as noted above, that the queue no longer is a threading
boundary. That, btw, is one reason I think threads do not belong into that
diagram: they are not related to the data flow, they are just helper entities
to implement it.
> also show where the message formatting take place (again, including
> thread is doing the work).
In the Action Postprocessor
> it is esecially unclear what happens where
> you have the action queues (is the message formatted before it's put
> the action queue, or is the format of a message in the action queue
> exactly the same as in the main queue?)
> some of this was covered (to at least some extent) by your comments,
> this sort of thing is actually covered better with text. I'm not a
> powerpoint person, but that sort of slideshow (with the text
> is ideal for this sort of explination.
Full ack, the tutorial is a "better than nothing" approach. Consolidating all
doc available (did you read the queue doc? If not, you really should) is a
major writing task, I'd think a couple of days (and reviews) to fully get it
> David Lang
> rsyslog mailing list
More information about the rsyslog