[rsyslog-notify] Forum Thread: Re: ompipe(action failed) or omfile( queues are filling) - (Mode 'reply')

[noreply at adiscon.com]

User: dlang 
Forumlink: http://kb.monitorware.com/viewtopic.php?p=26559#p26559

Message: 
----------
If you have multiple actions inside a ruleset that could block
independently, then it may make sense to give each of them a separate
queue. Just be aware that this is likely to end up eating CPU and limiting
your performance.

adding buffers isn't going to solve the problem if the consumer of the logs
isn't keeping up with the generator of the logs. The only thing that will
solve that problem is increasing the speed of the thing processing the
logs. buffers can help if you have temporary bursts above what the
processing can handle, but not if it's sustained above that level.

As for threads/batches.

Rsyslog receives messages and puts them into a queue. It then has a worker
thread read that queue and process however many messages are there, up to
the max batch size. This thread goes through all the actions and delivers
messages to all those destinations. If an action has it's own queue,
"delivering the message" just consists of putting it into that queue.

pulling a message from a queue or putting a message into a queue requires
locking the queue. So if you are reading from a queue and all you are doing
is inserting it into another queue, there is very little work that the
worker thread can do. As a result it is only going to end up processing one
message at a time rather than a batch of messages at a time. This means
that to process the same number of messages, the queues are going to be
locked a lot more of the time.

You can have more than one worker thread for any queue, but unless your
actions take a significant amount of time to process, you are likely to
find that the end result is actually worse performance because the threads
end up fighting each other for access to the queue.

The bottom line is that you should only increase threads if monitoring
shows that a thread is using close to 100% of a cpu.

you should only add a queue to an action/ruleset if you need to either have
another thread working on this subset of actions, or if the action can
block.