[rsyslog-notify] Forum Thread: Re: Please help: how to throttle remote logging? - (Mode 'reply')

[noreply at adiscon.com]

User: dlang 
Forumlink: http://kb.monitorware.com/viewtopic.php?p=25000#p25000

Message: 
----------
Yes, the best way to ratelimit output is really to do so at the network
layer where you limit the total bytes sent (so you don't send 100 30
character logs one minute and 100 4K character logs the next), and for that
HTB or iptables filters are really the right thing to do.

If you want to do it in rsyslog, you will need to take the dequeuebatchsize
into account when setting your limit.

so if you want <100 logs a second, you could do:

batch size of 1 and a delay of 1/100 of a second
batch size of 10 and a delay of 1/10 of a second
batch size of 100 and a delay of 1 second

what you did was to have a delay of 1/250 of a second, but you didn't limit
the batch size, which I think defaults to 128 or larger in current versions
(it was 16 in an early version with batch support), so instead of a max of
250 logs/sec, you hvae a max of 250 * 128 = ~32K logs/sec max. Since your
log volume is below that, you didn't see any change in total throughput. If
you were looking at a very granular view of the times that the logs
arrived, you would have seen that they started arriving in bursts instead
of being spread normally, but since you are talkng 250 bursts/sec you
probably would not have noticed this.

Does this make what's happening clearer?