[rsyslog] memory alloc problems - any advise?

[Rainer Gerhards]

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
> Sent: Monday, June 22, 2009 1:11 PM
> To: rsyslog-users
> Subject: Re: [rsyslog] memory alloc problems - any advise?
> My test driver produces almost identical messages, with mostly the same
size.
> I have not yet implemented any randomization into it. So in theory, it
should
> be fairly easy for the malloc() subsystem to reuse freed memory.
> 
> Anyhow, based on the malloc stats I got, this really looks like an issue
with
> the allocator itself. I'll for now implement an alloc_trim() every 100,000
> messages or so, that should keep care of this anomaly. Interestingly, v4
now
> does fewer allocs than v3. It almost looks like the alloc subsystem also
has
> a periodic "garbage collection" counter, which now seems to be triggered
less
> often.

Just FYI, I have implemented this algorithm now and it works pretty well.
Also, if there is an impact on performance, it is barely noticeable. 

I have also fixed a memory leak in v5 today, and run the same tests with v5
that I ran against v4. It is really impressive how much better it performs
(factor two, at least). On my low-end lab virtual machine, I am currently
processing around 150,00 tcp msgs/s (around 70,000 with v4) and this is also
close to sustained rate (that probably is around 10% below for v5).

Just thought I let you know.

Rainer