[rsyslog] Some list templates with mmjsonparse try to get 2GB of RAM or give OOM

Rainer Gerhards rgerhards at hq.adiscon.com
Mon Jan 7 15:16:40 CET 2013


Quick info: already got it, need to check how to handle it intelligently ;)

> -----Original Message-----
> From: rsyslog-bounces at lists.adiscon.com [mailto:rsyslog-
> bounces at lists.adiscon.com] On Behalf Of Radu Gheorghe
> Sent: Monday, January 07, 2013 2:46 PM
> To: rsyslog-users
> Subject: [rsyslog] Some list templates with mmjsonparse try to get 2GB
> of RAM or give OOM
> 
> Hello,
> 
> With the configuration below, rsyslog seems to try and allocate ~2GB of
> RAM
> (and quickly free that memory afterwards, I suppose) for each line of
> log
> it gets. So if my machine has 2GB of RAM or less, I get "**OUT OF
> MEMORY**"
> for my "$!all-json" property.
> 
> #####CONFIG START########
> module(load="imuxsock") # provides support for local system logging
> module(load="mmjsonparse")  #for parsing CEE-enhanced messages
> 
> template(name="testTemplate"
>          type="list") {
>              property(name="$!all-json" position.from="2")
>          }
> 
> *.* :mmjsonparse:
> 
> *.* action(type="omfile"
>            file="/tmp/logs"
>            template="testTemplate"
> )
> #####CONFIG END########
> 
> If I run it with valgrind, I get this for every log line:
> ==2436== Warning: set address range perms: large range [0x39431040,
> 0xb078a440) (undefined)
> ==2436== Warning: set address range perms: large range [0x39431030,
> 0xb078a450) (noaccess)
> 
> And after 3 logs, in the end I get this:
> ==2436==   total heap usage: 1,815 allocs, 1,773 frees, 6,000,203,586
> bytes
> allocated
> 
> However, if I remove the position.from part of the template, I get a
> much
> more reasonable:
> ==2441==   total heap usage: 1,805 allocs, 1,763 frees, 203,474 bytes
> allocated
> 
> I'm attaching both both valgrind outputs.
> 
> Another interesting thing is that the equivalent string template works
> fine:
> template(name="testTemplate" type="string"
>       string="%$!all-json:2:$::%"
> )
> 
> The problem is reproducible with rsyslog 7.2.4 and 7.3.5. I've tried on
> CentOS 6.3 and Ubuntu 12.04, both 64-bit. I've tried with the Ubuntu
> packages and also compiled sources manually - same thing.
> 
> Should I open a report in bugzilla about this?
> 
> Also, I have a related question: is it OK to use $!all-json in
> production?
> I don't see it in the property replacer documentation, and I was
> wondering
> if there's a reason for that.
> 
> And another question related to the related question: is there a way
> for
> contributing to the rsyslog official documentation (not wiki)? I'm
> particularly thinking about github pull requests - or maybe some
> similar
> system. I think it would be a nice way to keep the documentation
> updated.
> 
> Best regards,
> Radu


More information about the rsyslog mailing list