[rsyslog-notify] Forum Thread: Re: imtcp maxsessions - (Mode 'reply')

[noreply at adiscon.com]

User: dlang 
Forumlink: http://kb.monitorware.com/viewtopic.php?p=25675#p25675

Message: 
----------
just a note on architecture, I would probably tier the system a bit.
Instead of having all 800K clients talk to a single cluster of systems,
have the clients talk to a smaller cluster that's much closer to them, and
then have those relays forward to your central systems.

There ends up being a lot of work to do to clean up badly formatted logs
from various sources. There's value in spreading that work out to more
machines and simplifying your central ones.

Also, by going to local dedicated relays, you can then configure these
relays to send things to your central systems more efficiently (enable
compression for example)

I doubt if you have all 800K systems on a flat network, you probably have
them in groups (if not in different datacenters) and so the grouping of
what sends to what local relay set is probably fairly obvious.

Just something to think about.

David Lang

P.S. with this many client systems, what is the rate of logs that you get
at peak? (logs/sec and total bandwidth across all your systems would be
interesting)