From Jose.Castillo at ssa.gov  Tue Jul  9 17:21:38 2013
From: Jose.Castillo at ssa.gov (Castillo, Jose   Contractor)
Date: Tue, 9 Jul 2013 11:21:38 -0400
Subject: [Lognorm] log classification with liblognorm
In-Reply-To: <alpine.DEB.2.02.1306261357240.22572@nftneq.ynat.uz>
References: <701C784DE99B0C4C9FDA9EDE66E23E661E4CAD9300@HQ-MB-07.ba.ad.ssa.gov>
	<alpine.DEB.2.02.1306261210020.22572@nftneq.ynat.uz>
	<701C784DE99B0C4C9FDA9EDE66E23E661E4CAD930B@HQ-MB-07.ba.ad.ssa.gov>
	<alpine.DEB.2.02.1306261351570.22572@nftneq.ynat.uz>
	<alpine.DEB.2.02.1306261357240.22572@nftneq.ynat.uz>
Message-ID: <701C784DE99B0C4C9FDA9EDE66E23E661E4F593762@HQ-MB-07.ba.ad.ssa.gov>

Hello all,

Can anybody please tell me if (and how) rsyslog supports log classification with liblognorm (message classification tag, not the field tag)?

Thanks in advance.

Jose

-----Original Message-----
From: lognorm-bounces at lists.adiscon.com [mailto:lognorm-bounces at lists.adiscon.com] On Behalf Of David Lang
Sent: Wednesday, June 26, 2013 4:59 PM
To: lognorm
Subject: Re: [Lognorm] log classification with liblognorm

Ok, now I see what you are talking about. You are talking about the message classification tag, not the field tag. My mistake.

I don't know if and how rsyslog supports this.

David Lang

On Wed, 26 Jun 2013, David Lang wrote:

> First off, I have not used mmlognorm for anything other than seeing 
> that it works, so take that into consideration :-)
>
> However, as I'm looking at your rule, you have "mytag" outside of any 
> matching, so it would be treated as a literal, just like "Denied ICMP"
> wouldn't it?
>
> in your syslog template, try putting %$!number2% and see if it outputs 
> the correct value. it looks to me like that variable is being set correctly.
>
> David Lang
>
> On Wed, 26 Jun 2013, Castillo, Jose   Contractor wrote:
>
>> David,
>> 
>> I modified the template:
>> template(name="testFormat" type="string" 
>> string="%$!all-json%,tag='%$!mytag%'")
>> 
>> I also modified the rule to include mytag as a tag:
>> rule=mytag:%date:word% %host:ipv4% : 
>> %%ASA-%number0:number%-%number1:number%: Denied ICMP 
>> type=%number2:number%, code=%number3:number% from %origin:ipv4% on 
>> interface outside
>> 
>> And now I getting :
>> { "origin": "77.2.2.2", "number3": "0", "number2": "8", "number1": 
>> "313001", "number0": "3", "host": "0.0.0.0", "date": 
>> "2013-06-26T10:47:42+01:00" },tag=''
>> 
>> No tags are being written to the output file.
>> 
>> What am I doing wrong?
>> 
>> Thanks,
>> 
>> Jose
>> 
>> -----Original Message-----
>> From: lognorm-bounces at lists.adiscon.com 
>> [mailto:lognorm-bounces at lists.adiscon.com] On Behalf Of David Lang
>> Sent: Wednesday, June 26, 2013 3:11 PM
>> To: lognorm
>> Subject: Re: [Lognorm] log classification with liblognorm
>> 
>> the tags are accessed in rsyslog by $!<tag name>
>> 
>> so where you would have done %hostname% you could do %$!mytag%
>> 
>> David Lang
>> 
>> On Wed, 26 Jun 2013, Castillo, Jose   Contractor wrote:
>> 
>>> Date: Wed, 26 Jun 2013 16:00:06 -0400
>>> From: "Castillo, Jose   Contractor" 
>>> <Jose.Castillo at ssa.gov<mailto:Jose.Castillo at ssa.gov>>
>>> Reply-To: lognorm
>>> <lognorm at lists.adiscon.com<mailto:lognorm at lists.adiscon.com>>
>>> To: "lognorm at lists.adiscon.com<mailto:lognorm at lists.adiscon.com>" 
>>> <lognorm at lists.adiscon.com<mailto:lognorm at lists.adiscon.com>>
>>> Subject: [Lognorm] log classification with liblognorm
>>> 
>>> Hello all,
>>> 
>>> I started to play using rsyslog/liblognorm a week ago, I'm testing 
>>> them on a CentOS 6.4 virtual machine, and next packages have been 
>>> installed from Adiscon repository:
>>> rsyslog-mysql-7.4.1-1.el6.x86_64
>>> rsyslog-mmjsonparse-7.4.1-1.el6.x86_64
>>> rsyslog-7.4.1-1.el6.x86_64
>>> rsyslog-mmnormalize-7.4.1-1.el6.x86_64
>>> rsyslog-udpspoof-7.4.1-1.el6.x86_64
>>> rsyslog-elasticsearch-7.4.1-1.el6.x86_64
>>> 
>>> I was reading next link, and now I'm trying to select subsets of 
>>> messages based on liblognorm tags.
>>> http://blog.gerhards.net/2011/04/log-classification-with-liblognorm.
>>> ht
>>> ml
>>> 
>>> So far, I was able to parse a message using mmnormalize module, and 
>>> now I want to create outputs based on the liblognorm tag(s).
>>> Is this option supported within rsyslog.conf? Any example how to do it?
>>> 
>>> Below my testing files:
>>> 
>>> =================================================================
>>> rsyslog.conf file:
>>> module (load="imudp")
>>> module (load="mmnormalize")
>>> module (load="mmjsonparse")
>>> 
>>> input(type="imudp" address="192.168.1.1" port="514" ruleset="test")
>>> 
>>> template(name="testFormat" type="string" string="%$!all-json%\n")
>>> 
>>> ruleset(name="test") {
>>>        action(type="mmnormalize" userawmsg="on" 
>>> rulebase="/data/syslog/rulebase.rb")
>>>        action(type="omfile" file="/data/syslog/test-syslog.log"
>>> template="testFormat") }
>>> 
>>> rulebase.rb file:
>>> rule=test:%date:word% %host:ipv4% :
>>> %%ASA-%number0:number%-%number1:number%: Denied ICMP 
>>> type=%number2:number%, code=%number3:number% from %origin:ipv4% on 
>>> interface outside
>>> 
>>> test-syslog.log file:
>>> { "origin": "77.2.2.2", "number3": "0", "number2": "8", "number1":
>>> "313001", "number0": "3", "host": "0.0.0.0", "date":
>>> "2013-06-26T10:47:42+01:00" }
>>> 
>>> Message:
>>> "2013-06-26T10:47:42+01:00 0.0.0.0 : %ASA-3-313001: Denied ICMP 
>>> type=8,
>>> code=0 from 77.2.2.2 on interface outside"
>>> ====================================================================
>>> ==
>>> 
>>> 
>>> Thanks in advance for any help,
>>> 
>>> Jose Castillo
>>> 
>>> 
>>> 
>>> 
>> 
>

From cclark at quadrantsec.com  Fri Jul 12 22:00:17 2013
From: cclark at quadrantsec.com (Champ Clark III)
Date: Fri, 12 Jul 2013 16:00:17 -0400
Subject: [Lognorm] Memory Leak in liblognorm?
Message-ID: <51E06051.5040901@quadrantsec.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


Hello All!

I've got either a memory leak in liblognorm _or_ I'm not doing
something correctly.   In Sagan,  we have "processors" that do log
analysis using other techniques besides "rules".  As part of this
process,  liblognorm is used pretty heavily.

In the Sagan processor,  if I turn on liblognorm,  Sagan slowly starts
to consume memory.  However,  if I tell the processor not to use
liblognorm,  the memory stays consistent.

My thoughts are that I'm not either clean up after using liblognorm
correctly or liblognorm has a slow memory leak.

My slightly mangled debug/Sagan code is at:

https://github.com/beave/sagan/blob/master/src/sagan-liblognorm.c

I think the problem is in the sagan_normalize_liblognorm() function.
Near the end,  I've tried various ways to clean up, but with no
affect.  (ee_deleteEvent(), free(), etc)

Any ideas would be greatly appreciated.


- -- 
- - Champ Clark III (cclark at quadrantsec.com)
  Quadrant Information Security (http://quadrantsec.com)
  Key Fingerprint: 2E56 C2EB 1B25 C517 D5BA 2DCF 5E70 B2F8 0381 878A
  GPG Key ID: 0381878A
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJR4GBRAAoJENnmXt7Lmc3KpIIH/3RP5p/oW9BonTE9/7Pj5pro
SdijQuYGQOox2yuWKwxAXGzES5jMu5ItqH+cdi7Oi3ddtPEVVEgvPVcGgYqiWhDP
ic6APvPgG9nAPT/ouCwjDc1Wi4w6n/sAXJgAmqFjfjJA3Idxeqhr4XPcUyeDunc8
MkR/Nbrozoo7n3pNoZSqR9mpd//ERF1nwcSUjmtBzuLSiQSR9FCl9JktykW6E2MR
76fZd6prxTkhMigFC9NHF5Hf1B5ByrTOt5W76zzP2bwTK1XI5NDya0GKGixZMhEx
cCgApVAFA0eA0eL8uK+cIpKSd0tLNzsnH0GoR66mwhtqiQPoNkji5Og4+JrkowQ=
=JDfT
-----END PGP SIGNATURE-----

From rgerhards at hq.adiscon.com  Fri Jul 12 22:02:29 2013
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 12 Jul 2013 22:02:29 +0200
Subject: [Lognorm] Memory Leak in liblognorm?
In-Reply-To: <51E06051.5040901@quadrantsec.com>
References: <51E06051.5040901@quadrantsec.com>
Message-ID: <CADk+mPCMKNr+Be7LedbvvdCYnY=L432TMLcTav+wcJpUb+nfKA@mail.gmail.com>

Can you run it under valgrind and tell what it says at end of run?

Sent from phone, thus brief.
Am 12.07.2013 22:00 schrieb "Champ Clark III" <cclark at quadrantsec.com>:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
> Hello All!
>
> I've got either a memory leak in liblognorm _or_ I'm not doing
> something correctly.   In Sagan,  we have "processors" that do log
> analysis using other techniques besides "rules".  As part of this
> process,  liblognorm is used pretty heavily.
>
> In the Sagan processor,  if I turn on liblognorm,  Sagan slowly starts
> to consume memory.  However,  if I tell the processor not to use
> liblognorm,  the memory stays consistent.
>
> My thoughts are that I'm not either clean up after using liblognorm
> correctly or liblognorm has a slow memory leak.
>
> My slightly mangled debug/Sagan code is at:
>
> https://github.com/beave/sagan/blob/master/src/sagan-liblognorm.c
>
> I think the problem is in the sagan_normalize_liblognorm() function.
> Near the end,  I've tried various ways to clean up, but with no
> affect.  (ee_deleteEvent(), free(), etc)
>
> Any ideas would be greatly appreciated.
>
>
> - --
> - - Champ Clark III (cclark at quadrantsec.com)
>   Quadrant Information Security (http://quadrantsec.com)
>   Key Fingerprint: 2E56 C2EB 1B25 C517 D5BA 2DCF 5E70 B2F8 0381 878A
>   GPG Key ID: 0381878A
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQEcBAEBAgAGBQJR4GBRAAoJENnmXt7Lmc3KpIIH/3RP5p/oW9BonTE9/7Pj5pro
> SdijQuYGQOox2yuWKwxAXGzES5jMu5ItqH+cdi7Oi3ddtPEVVEgvPVcGgYqiWhDP
> ic6APvPgG9nAPT/ouCwjDc1Wi4w6n/sAXJgAmqFjfjJA3Idxeqhr4XPcUyeDunc8
> MkR/Nbrozoo7n3pNoZSqR9mpd//ERF1nwcSUjmtBzuLSiQSR9FCl9JktykW6E2MR
> 76fZd6prxTkhMigFC9NHF5Hf1B5ByrTOt5W76zzP2bwTK1XI5NDya0GKGixZMhEx
> cCgApVAFA0eA0eL8uK+cIpKSd0tLNzsnH0GoR66mwhtqiQPoNkji5Og4+JrkowQ=
> =JDfT
> -----END PGP SIGNATURE-----
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20130712/6e3ed5ae/attachment.htm>

From cclark at quadrantsec.com  Fri Jul 12 22:05:27 2013
From: cclark at quadrantsec.com (Champ Clark III)
Date: Fri, 12 Jul 2013 16:05:27 -0400
Subject: [Lognorm] Memory Leak in liblognorm?
In-Reply-To: <CADk+mPCMKNr+Be7LedbvvdCYnY=L432TMLcTav+wcJpUb+nfKA@mail.gmail.com>
References: <51E06051.5040901@quadrantsec.com>
	<CADk+mPCMKNr+Be7LedbvvdCYnY=L432TMLcTav+wcJpUb+nfKA@mail.gmail.com>
Message-ID: <51E06187.2010908@quadrantsec.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I'll have to run on a different box,  as this machine (for whatever
reason) doesn't play well with valgrind.  I'll do that from a Ubuntu
VM ASAP.

If you get a chance,  can you glance and the code and see if I'm
blatantly and improperly cleaning up incorrectly?

I'll let you know the results from valgrind ASAP.


On 7/12/13 4:02 PM, Rainer Gerhards wrote:
> Can you run it under valgrind and tell what it says at end of run?
> 
> Sent from phone, thus brief.
> 
> Am 12.07.2013 22:00 schrieb "Champ Clark III"
> <cclark at quadrantsec.com <mailto:cclark at quadrantsec.com>>:
> 
> 
> Hello All!
> 
> I've got either a memory leak in liblognorm _or_ I'm not doing 
> something correctly.   In Sagan,  we have "processors" that do log 
> analysis using other techniques besides "rules".  As part of this 
> process,  liblognorm is used pretty heavily.
> 
> In the Sagan processor,  if I turn on liblognorm,  Sagan slowly
> starts to consume memory.  However,  if I tell the processor not to
> use liblognorm,  the memory stays consistent.
> 
> My thoughts are that I'm not either clean up after using
> liblognorm correctly or liblognorm has a slow memory leak.
> 
> My slightly mangled debug/Sagan code is at:
> 
> https://github.com/beave/sagan/blob/master/src/sagan-liblognorm.c
> 
> I think the problem is in the sagan_normalize_liblognorm()
> function. Near the end,  I've tried various ways to clean up, but
> with no affect.  (ee_deleteEvent(), free(), etc)
> 
> Any ideas would be greatly appreciated.
> 
> 
> _______________________________________________ Lognorm mailing
> list Lognorm at lists.adiscon.com <mailto:Lognorm at lists.adiscon.com> 
> http://lists.adiscon.net/mailman/listinfo/lognorm
> 
> 
> 
> _______________________________________________ Lognorm mailing
> list Lognorm at lists.adiscon.com 
> http://lists.adiscon.net/mailman/listinfo/lognorm
> 

- -- 
- - Champ Clark III (cclark at quadrantsec.com)
  Quadrant Information Security (http://quadrantsec.com)
  Key Fingerprint: 2E56 C2EB 1B25 C517 D5BA 2DCF 5E70 B2F8 0381 878A
  GPG Key ID: 0381878A
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJR4GGHAAoJENnmXt7Lmc3K4oMH+wRHHkchFSsVgSx/CRZ33Ae3
YAxqJNVKwNNkiS6wRKnMJp+odoV804VxxRzotSEFGHu3Tiz59p9cEPYf9SUK6hve
zWONr3TdeCHkOaXZweeOZT5LK+Q+acehjIZp3314WFgm+NAfJ7Ms6fX/VC4qi/Yh
MgFPlBjYjqYz4xAQRFvS3LtzU7U0mA+FXhT0LpchvfyDMKkSLmhJ+I3S+7qLExJo
YnmI5Bt5laIVc4KAC8rzC/uKLuLRdaXbxrNUTqCZebLDdwJ040Llg267MdAZBiDR
5kQXXqH3a6tykebosydtZUxbp5OrXUVoVmOBZrGQYsQs4bb6P8YLKNtlRImADLA=
=x/xI
-----END PGP SIGNATURE-----

From rgerhards at hq.adiscon.com  Fri Jul 12 22:20:17 2013
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 12 Jul 2013 22:20:17 +0200
Subject: [Lognorm] Memory Leak in liblognorm?
In-Reply-To: <51E06187.2010908@quadrantsec.com>
References: <51E06051.5040901@quadrantsec.com>
	<CADk+mPCMKNr+Be7LedbvvdCYnY=L432TMLcTav+wcJpUb+nfKA@mail.gmail.com>
	<51E06187.2010908@quadrantsec.com>
Message-ID: <CADk+mPBULO7gdETi-Wo3RgPFXBzwyvTNBE_E_yOVdAbtj_qF3A@mail.gmail.com>

Will do tomorrow

Sent from phone, thus brief.
Am 12.07.2013 22:05 schrieb "Champ Clark III" <cclark at quadrantsec.com>:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> I'll have to run on a different box,  as this machine (for whatever
> reason) doesn't play well with valgrind.  I'll do that from a Ubuntu
> VM ASAP.
>
> If you get a chance,  can you glance and the code and see if I'm
> blatantly and improperly cleaning up incorrectly?
>
> I'll let you know the results from valgrind ASAP.
>
>
> On 7/12/13 4:02 PM, Rainer Gerhards wrote:
> > Can you run it under valgrind and tell what it says at end of run?
> >
> > Sent from phone, thus brief.
> >
> > Am 12.07.2013 22:00 schrieb "Champ Clark III"
> > <cclark at quadrantsec.com <mailto:cclark at quadrantsec.com>>:
> >
> >
> > Hello All!
> >
> > I've got either a memory leak in liblognorm _or_ I'm not doing
> > something correctly.   In Sagan,  we have "processors" that do log
> > analysis using other techniques besides "rules".  As part of this
> > process,  liblognorm is used pretty heavily.
> >
> > In the Sagan processor,  if I turn on liblognorm,  Sagan slowly
> > starts to consume memory.  However,  if I tell the processor not to
> > use liblognorm,  the memory stays consistent.
> >
> > My thoughts are that I'm not either clean up after using
> > liblognorm correctly or liblognorm has a slow memory leak.
> >
> > My slightly mangled debug/Sagan code is at:
> >
> > https://github.com/beave/sagan/blob/master/src/sagan-liblognorm.c
> >
> > I think the problem is in the sagan_normalize_liblognorm()
> > function. Near the end,  I've tried various ways to clean up, but
> > with no affect.  (ee_deleteEvent(), free(), etc)
> >
> > Any ideas would be greatly appreciated.
> >
> >
> > _______________________________________________ Lognorm mailing
> > list Lognorm at lists.adiscon.com <mailto:Lognorm at lists.adiscon.com>
> > http://lists.adiscon.net/mailman/listinfo/lognorm
> >
> >
> >
> > _______________________________________________ Lognorm mailing
> > list Lognorm at lists.adiscon.com
> > http://lists.adiscon.net/mailman/listinfo/lognorm
> >
>
> - --
> - - Champ Clark III (cclark at quadrantsec.com)
>   Quadrant Information Security (http://quadrantsec.com)
>   Key Fingerprint: 2E56 C2EB 1B25 C517 D5BA 2DCF 5E70 B2F8 0381 878A
>   GPG Key ID: 0381878A
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQEcBAEBAgAGBQJR4GGHAAoJENnmXt7Lmc3K4oMH+wRHHkchFSsVgSx/CRZ33Ae3
> YAxqJNVKwNNkiS6wRKnMJp+odoV804VxxRzotSEFGHu3Tiz59p9cEPYf9SUK6hve
> zWONr3TdeCHkOaXZweeOZT5LK+Q+acehjIZp3314WFgm+NAfJ7Ms6fX/VC4qi/Yh
> MgFPlBjYjqYz4xAQRFvS3LtzU7U0mA+FXhT0LpchvfyDMKkSLmhJ+I3S+7qLExJo
> YnmI5Bt5laIVc4KAC8rzC/uKLuLRdaXbxrNUTqCZebLDdwJ040Llg267MdAZBiDR
> 5kQXXqH3a6tykebosydtZUxbp5OrXUVoVmOBZrGQYsQs4bb6P8YLKNtlRImADLA=
> =x/xI
> -----END PGP SIGNATURE-----
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20130712/be202165/attachment-0001.htm>

From zinic at jpserver.net  Fri Jul 12 22:27:42 2013
From: zinic at jpserver.net (John Hopper)
Date: Fri, 12 Jul 2013 15:27:42 -0500
Subject: [Lognorm] Memory Leak in liblognorm?
In-Reply-To: <CADk+mPBULO7gdETi-Wo3RgPFXBzwyvTNBE_E_yOVdAbtj_qF3A@mail.gmail.com>
References: <51E06051.5040901@quadrantsec.com>
	<CADk+mPCMKNr+Be7LedbvvdCYnY=L432TMLcTav+wcJpUb+nfKA@mail.gmail.com>
	<51E06187.2010908@quadrantsec.com>
	<CADk+mPBULO7gdETi-Wo3RgPFXBzwyvTNBE_E_yOVdAbtj_qF3A@mail.gmail.com>
Message-ID: <51E066BE.2010804@jpserver.net>

Heya!

I noticed in your code that you're calling the estr and ee library to
create events and strings but I don't see any calls to the delete
functions for either.

As far as I know, every time you create a new estr, it copies the
content into a new buffer which will need to be freed. The same applies
for the events created - these will also need to be freed.

-John

On 07/12/2013 03:20 PM, Rainer Gerhards wrote:
> Will do tomorrow
> 
> Sent from phone, thus brief.
> Am 12.07.2013 22:05 schrieb "Champ Clark III" <cclark at quadrantsec.com>:
> 
> I'll have to run on a different box,  as this machine (for whatever
> reason) doesn't play well with valgrind.  I'll do that from a Ubuntu
> VM ASAP.
> 
> If you get a chance,  can you glance and the code and see if I'm
> blatantly and improperly cleaning up incorrectly?
> 
> I'll let you know the results from valgrind ASAP.
> 
> 
> On 7/12/13 4:02 PM, Rainer Gerhards wrote:
>>>> Can you run it under valgrind and tell what it says at end of run?
>>>>
>>>> Sent from phone, thus brief.
>>>>
>>>> Am 12.07.2013 22:00 schrieb "Champ Clark III"
>>>> <cclark at quadrantsec.com <mailto:cclark at quadrantsec.com>>:
>>>>
>>>>
>>>> Hello All!
>>>>
>>>> I've got either a memory leak in liblognorm _or_ I'm not doing
>>>> something correctly.   In Sagan,  we have "processors" that do log
>>>> analysis using other techniques besides "rules".  As part of this
>>>> process,  liblognorm is used pretty heavily.
>>>>
>>>> In the Sagan processor,  if I turn on liblognorm,  Sagan slowly
>>>> starts to consume memory.  However,  if I tell the processor not to
>>>> use liblognorm,  the memory stays consistent.
>>>>
>>>> My thoughts are that I'm not either clean up after using
>>>> liblognorm correctly or liblognorm has a slow memory leak.
>>>>
>>>> My slightly mangled debug/Sagan code is at:
>>>>
>>>> https://github.com/beave/sagan/blob/master/src/sagan-liblognorm.c
>>>>
>>>> I think the problem is in the sagan_normalize_liblognorm()
>>>> function. Near the end,  I've tried various ways to clean up, but
>>>> with no affect.  (ee_deleteEvent(), free(), etc)
>>>>
>>>> Any ideas would be greatly appreciated.
>>>>
>>>>
>>>> _______________________________________________ Lognorm mailing
>>>> list Lognorm at lists.adiscon.com <mailto:Lognorm at lists.adiscon.com>
>>>> http://lists.adiscon.net/mailman/listinfo/lognorm
>>>>
>>>>
>>>>
>>>> _______________________________________________ Lognorm mailing
>>>> list Lognorm at lists.adiscon.com
>>>> http://lists.adiscon.net/mailman/listinfo/lognorm
>>>>
> 
>> _______________________________________________
>> Lognorm mailing list
>> Lognorm at lists.adiscon.com
>> http://lists.adiscon.net/mailman/listinfo/lognorm
>>
> 
> 
> 
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm
> 

From cclark at quadrantsec.com  Sat Jul 13 13:39:01 2013
From: cclark at quadrantsec.com (Champ Clark III)
Date: Sat, 13 Jul 2013 07:39:01 -0400
Subject: [Lognorm] Memory Leak in liblognorm?
In-Reply-To: <51E066BE.2010804@jpserver.net>
References: <51E06051.5040901@quadrantsec.com>
	<CADk+mPCMKNr+Be7LedbvvdCYnY=L432TMLcTav+wcJpUb+nfKA@mail.gmail.com>
	<51E06187.2010908@quadrantsec.com>
	<CADk+mPBULO7gdETi-Wo3RgPFXBzwyvTNBE_E_yOVdAbtj_qF3A@mail.gmail.com>
	<51E066BE.2010804@jpserver.net>
Message-ID: <51E13C55.5000401@quadrantsec.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Thank you John,

I've added a es_deleteStr(str) and freed just about all I can think of
and I'm seeing the same results.  Just to be on the safe side,  I ran
two versions of Sagan last night.  One using liblognorm and the other
without.  I let these run for about 8 hours.

In the end,  the one without liblognorm used about 2.5 meg of RAM.
The one with liblognorm eventually crept up to 128 meg of RAM (and
still climbing).

Like last time,  I pushed the code up to github
(https://github.com/beave/sagan/blob/master/src/sagan-liblognorm.c).
You where right about the es_deleteStr,  but it's still slowly
leaking.  Essentially, this is what I did:

free(cstr);
free(field);
es_deleteStr(str);
es_deleteStr(propName);		/* Just to be safe? */
ee_deleteEvent(lnevent);


I should add that I'm using the liblognorm (and other require libs)
from Rainer's git repo.

I got my Ubuntu 12.04 box back up with the dev tools and valgrind last
night,  but haven't had a chance to move over Sagan and test yet.   I
hope to be able to do that today.

I feel that I'm not handling something correctly,  so if you spot it
let me know!  I really appreciate it.





On 7/12/13 4:27 PM, John Hopper wrote:
> Heya!
> 
> I noticed in your code that you're calling the estr and ee library
> to create events and strings but I don't see any calls to the
> delete functions for either.
> 
> As far as I know, every time you create a new estr, it copies the 
> content into a new buffer which will need to be freed. The same
> applies for the events created - these will also need to be freed.
> 
> -John
> 

- -- 
- - Champ Clark III (cclark at quadrantsec.com)
  Quadrant Information Security (http://quadrantsec.com)
  Key Fingerprint: 2E56 C2EB 1B25 C517 D5BA 2DCF 5E70 B2F8 0381 878A
  GPG Key ID: 0381878A
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJR4TxUAAoJENnmXt7Lmc3KnuEH/0JzCAiG9oMdtzp7KMejAkJh
FbtJBVCq+vIHSs9LMmJPTgRkekZBG+DhobMp9sHDvr01Hh8fAFbKvuQAS0Qu5T8t
ukxfoF2Ia0C9VPJttiZrMEtKpkAk9vRNpYGad9DdJWrrfh4VcX9zwXp2x32CBJlC
ehXyjE4UqN8SfHtBY1XP1WR3YqitplN9HjR7FW48rxHcraNleFtn442mH/2cXJZ1
+IllY7MoJf/VdoXWKCZzQhEQjdNek4bTxt0K9cdDg/TwD4yucm9XMO+lw1pSluBX
4Vkh1XCFlQsgWsG5Vf5pQPJdTx0qfOj3mRspz2v/UocnM0iv9aAA5iNEkn5Ouu4=
=qMCp
-----END PGP SIGNATURE-----

From rgerhards at hq.adiscon.com  Sat Jul 13 16:09:14 2013
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Sat, 13 Jul 2013 16:09:14 +0200
Subject: [Lognorm] Memory Leak in liblognorm?
In-Reply-To: <51E13C55.5000401@quadrantsec.com>
References: <51E06051.5040901@quadrantsec.com>
	<CADk+mPCMKNr+Be7LedbvvdCYnY=L432TMLcTav+wcJpUb+nfKA@mail.gmail.com>
	<51E06187.2010908@quadrantsec.com>
	<CADk+mPBULO7gdETi-Wo3RgPFXBzwyvTNBE_E_yOVdAbtj_qF3A@mail.gmail.com>
	<51E066BE.2010804@jpserver.net> <51E13C55.5000401@quadrantsec.com>
Message-ID: <CADk+mPDYAQ_4KvU26W9FTogp4NAW8vbYbWguXrpi7yzxAW3pjw@mail.gmail.com>

any chance to get a valgrind log? That would be immensely helpful, as it
usually points out where memory was allocted...


On Sat, Jul 13, 2013 at 1:39 PM, Champ Clark III <cclark at quadrantsec.com>wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Thank you John,
>
> I've added a es_deleteStr(str) and freed just about all I can think of
> and I'm seeing the same results.  Just to be on the safe side,  I ran
> two versions of Sagan last night.  One using liblognorm and the other
> without.  I let these run for about 8 hours.
>
> In the end,  the one without liblognorm used about 2.5 meg of RAM.
> The one with liblognorm eventually crept up to 128 meg of RAM (and
> still climbing).
>
> Like last time,  I pushed the code up to github
> (https://github.com/beave/sagan/blob/master/src/sagan-liblognorm.c).
> You where right about the es_deleteStr,  but it's still slowly
> leaking.  Essentially, this is what I did:
>
> free(cstr);
> free(field);
> es_deleteStr(str);
> es_deleteStr(propName);         /* Just to be safe? */
> ee_deleteEvent(lnevent);
>
>
> I should add that I'm using the liblognorm (and other require libs)
> from Rainer's git repo.
>
> I got my Ubuntu 12.04 box back up with the dev tools and valgrind last
> night,  but haven't had a chance to move over Sagan and test yet.   I
> hope to be able to do that today.
>
> I feel that I'm not handling something correctly,  so if you spot it
> let me know!  I really appreciate it.
>
>
>
>
>
> On 7/12/13 4:27 PM, John Hopper wrote:
> > Heya!
> >
> > I noticed in your code that you're calling the estr and ee library
> > to create events and strings but I don't see any calls to the
> > delete functions for either.
> >
> > As far as I know, every time you create a new estr, it copies the
> > content into a new buffer which will need to be freed. The same
> > applies for the events created - these will also need to be freed.
> >
> > -John
> >
>
> - --
> - - Champ Clark III (cclark at quadrantsec.com)
>   Quadrant Information Security (http://quadrantsec.com)
>   Key Fingerprint: 2E56 C2EB 1B25 C517 D5BA 2DCF 5E70 B2F8 0381 878A
>   GPG Key ID: 0381878A
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQEcBAEBAgAGBQJR4TxUAAoJENnmXt7Lmc3KnuEH/0JzCAiG9oMdtzp7KMejAkJh
> FbtJBVCq+vIHSs9LMmJPTgRkekZBG+DhobMp9sHDvr01Hh8fAFbKvuQAS0Qu5T8t
> ukxfoF2Ia0C9VPJttiZrMEtKpkAk9vRNpYGad9DdJWrrfh4VcX9zwXp2x32CBJlC
> ehXyjE4UqN8SfHtBY1XP1WR3YqitplN9HjR7FW48rxHcraNleFtn442mH/2cXJZ1
> +IllY7MoJf/VdoXWKCZzQhEQjdNek4bTxt0K9cdDg/TwD4yucm9XMO+lw1pSluBX
> 4Vkh1XCFlQsgWsG5Vf5pQPJdTx0qfOj3mRspz2v/UocnM0iv9aAA5iNEkn5Ouu4=
> =qMCp
> -----END PGP SIGNATURE-----
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20130713/b4e81365/attachment.htm>

From rgerhards at hq.adiscon.com  Sat Jul 13 16:13:39 2013
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Sat, 13 Jul 2013 16:13:39 +0200
Subject: [Lognorm] Memory Leak in liblognorm?
In-Reply-To: <CADk+mPDYAQ_4KvU26W9FTogp4NAW8vbYbWguXrpi7yzxAW3pjw@mail.gmail.com>
References: <51E06051.5040901@quadrantsec.com>
	<CADk+mPCMKNr+Be7LedbvvdCYnY=L432TMLcTav+wcJpUb+nfKA@mail.gmail.com>
	<51E06187.2010908@quadrantsec.com>
	<CADk+mPBULO7gdETi-Wo3RgPFXBzwyvTNBE_E_yOVdAbtj_qF3A@mail.gmail.com>
	<51E066BE.2010804@jpserver.net> <51E13C55.5000401@quadrantsec.com>
	<CADk+mPDYAQ_4KvU26W9FTogp4NAW8vbYbWguXrpi7yzxAW3pjw@mail.gmail.com>
Message-ID: <CADk+mPAeLW9rTeLOjheK47wgWCZHF7AzKbyhL-bdcrC8B_Xa6w@mail.gmail.com>

On Sat, Jul 13, 2013 at 4:09 PM, Rainer Gerhards
<rgerhards at hq.adiscon.com>wrote:

> any chance to get a valgrind log? That would be immensely helpful, as it
> usually points out where memory was allocted...
>
>
I should probably add that I have given up finding such problems from
looking at code, because I tend to always overlook something. In contrast,
the valgrind log usually gives a very precise clue of where the problem is.
For my own code, I *always* use valgrind and am very unhappy if that
doesn't do the trick. Strongly boosted my productivity. That's why I am so
insistive on getting the valgrind log.

I have quickly looked at your code, but I am sure I'll miss something or
need to look very precisely and paper&pencil-track alloc vs. free... That
quick look didn't show anything, though. But if it is in liblognorm, I
would need to repro over here, so that I can look at it ... with valgrind ;)

Rainer

>
> On Sat, Jul 13, 2013 at 1:39 PM, Champ Clark III <cclark at quadrantsec.com>wrote:
>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Thank you John,
>>
>> I've added a es_deleteStr(str) and freed just about all I can think of
>> and I'm seeing the same results.  Just to be on the safe side,  I ran
>> two versions of Sagan last night.  One using liblognorm and the other
>> without.  I let these run for about 8 hours.
>>
>> In the end,  the one without liblognorm used about 2.5 meg of RAM.
>> The one with liblognorm eventually crept up to 128 meg of RAM (and
>> still climbing).
>>
>> Like last time,  I pushed the code up to github
>> (https://github.com/beave/sagan/blob/master/src/sagan-liblognorm.c).
>> You where right about the es_deleteStr,  but it's still slowly
>> leaking.  Essentially, this is what I did:
>>
>> free(cstr);
>> free(field);
>> es_deleteStr(str);
>> es_deleteStr(propName);         /* Just to be safe? */
>> ee_deleteEvent(lnevent);
>>
>>
>> I should add that I'm using the liblognorm (and other require libs)
>> from Rainer's git repo.
>>
>> I got my Ubuntu 12.04 box back up with the dev tools and valgrind last
>> night,  but haven't had a chance to move over Sagan and test yet.   I
>> hope to be able to do that today.
>>
>> I feel that I'm not handling something correctly,  so if you spot it
>> let me know!  I really appreciate it.
>>
>>
>>
>>
>>
>> On 7/12/13 4:27 PM, John Hopper wrote:
>> > Heya!
>> >
>> > I noticed in your code that you're calling the estr and ee library
>> > to create events and strings but I don't see any calls to the
>> > delete functions for either.
>> >
>> > As far as I know, every time you create a new estr, it copies the
>> > content into a new buffer which will need to be freed. The same
>> > applies for the events created - these will also need to be freed.
>> >
>> > -John
>> >
>>
>> - --
>> - - Champ Clark III (cclark at quadrantsec.com)
>>   Quadrant Information Security (http://quadrantsec.com)
>>   Key Fingerprint: 2E56 C2EB 1B25 C517 D5BA 2DCF 5E70 B2F8 0381 878A
>>   GPG Key ID: 0381878A
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
>> Comment: GPGTools - http://gpgtools.org
>> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>>
>> iQEcBAEBAgAGBQJR4TxUAAoJENnmXt7Lmc3KnuEH/0JzCAiG9oMdtzp7KMejAkJh
>> FbtJBVCq+vIHSs9LMmJPTgRkekZBG+DhobMp9sHDvr01Hh8fAFbKvuQAS0Qu5T8t
>> ukxfoF2Ia0C9VPJttiZrMEtKpkAk9vRNpYGad9DdJWrrfh4VcX9zwXp2x32CBJlC
>> ehXyjE4UqN8SfHtBY1XP1WR3YqitplN9HjR7FW48rxHcraNleFtn442mH/2cXJZ1
>> +IllY7MoJf/VdoXWKCZzQhEQjdNek4bTxt0K9cdDg/TwD4yucm9XMO+lw1pSluBX
>> 4Vkh1XCFlQsgWsG5Vf5pQPJdTx0qfOj3mRspz2v/UocnM0iv9aAA5iNEkn5Ouu4=
>> =qMCp
>> -----END PGP SIGNATURE-----
>> _______________________________________________
>> Lognorm mailing list
>> Lognorm at lists.adiscon.com
>> http://lists.adiscon.net/mailman/listinfo/lognorm
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20130713/6bb49f24/attachment-0001.htm>

From cclark at quadrantsec.com  Sat Jul 13 16:31:55 2013
From: cclark at quadrantsec.com (Champ Clark III)
Date: Sat, 13 Jul 2013 10:31:55 -0400
Subject: [Lognorm] Memory Leak in liblognorm?
In-Reply-To: <CADk+mPDYAQ_4KvU26W9FTogp4NAW8vbYbWguXrpi7yzxAW3pjw@mail.gmail.com>
References: <51E06051.5040901@quadrantsec.com>
	<CADk+mPCMKNr+Be7LedbvvdCYnY=L432TMLcTav+wcJpUb+nfKA@mail.gmail.com>
	<51E06187.2010908@quadrantsec.com>
	<CADk+mPBULO7gdETi-Wo3RgPFXBzwyvTNBE_E_yOVdAbtj_qF3A@mail.gmail.com>
	<51E066BE.2010804@jpserver.net> <51E13C55.5000401@quadrantsec.com>
	<CADk+mPDYAQ_4KvU26W9FTogp4NAW8vbYbWguXrpi7yzxAW3pjw@mail.gmail.com>
Message-ID: <5b0c7475-2516-412c-9460-02cf1d82ad1d@email.android.com>

I understand about valgrind..   from my laet email is below... ill have you that log asap.


"I got my Ubuntu 12.04 box back up with the dev tools and valgrind last
night,  but haven't had a chance to move over Sagan and test yet.   I
hope to be able to do that today."

Rainer Gerhards <rgerhards at hq.adiscon.com> wrote:

>any chance to get a valgrind log? That would be immensely helpful, as
>it
>usually points out where memory was allocted...
>
>
>On Sat, Jul 13, 2013 at 1:39 PM, Champ Clark III
><cclark at quadrantsec.com>wrote:
>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Thank you John,
>>
>> I've added a es_deleteStr(str) and freed just about all I can think
>of
>> and I'm seeing the same results.  Just to be on the safe side,  I ran
>> two versions of Sagan last night.  One using liblognorm and the other
>> without.  I let these run for about 8 hours.
>>
>> In the end,  the one without liblognorm used about 2.5 meg of RAM.
>> The one with liblognorm eventually crept up to 128 meg of RAM (and
>> still climbing).
>>
>> Like last time,  I pushed the code up to github
>> (https://github.com/beave/sagan/blob/master/src/sagan-liblognorm.c).
>> You where right about the es_deleteStr,  but it's still slowly
>> leaking.  Essentially, this is what I did:
>>
>> free(cstr);
>> free(field);
>> es_deleteStr(str);
>> es_deleteStr(propName);         /* Just to be safe? */
>> ee_deleteEvent(lnevent);
>>
>>
>> I should add that I'm using the liblognorm (and other require libs)
>> from Rainer's git repo.
>>
>> I got my Ubuntu 12.04 box back up with the dev tools and valgrind
>last
>> night,  but haven't had a chance to move over Sagan and test yet.   I
>> hope to be able to do that today.
>>
>> I feel that I'm not handling something correctly,  so if you spot it
>> let me know!  I really appreciate it.
>>
>>
>>
>>
>>
>> On 7/12/13 4:27 PM, John Hopper wrote:
>> > Heya!
>> >
>> > I noticed in your code that you're calling the estr and ee library
>> > to create events and strings but I don't see any calls to the
>> > delete functions for either.
>> >
>> > As far as I know, every time you create a new estr, it copies the
>> > content into a new buffer which will need to be freed. The same
>> > applies for the events created - these will also need to be freed.
>> >
>> > -John
>> >
>>
>> - --
>> - - Champ Clark III (cclark at quadrantsec.com)
>>   Quadrant Information Security (http://quadrantsec.com)
>>   Key Fingerprint: 2E56 C2EB 1B25 C517 D5BA 2DCF 5E70 B2F8 0381 878A
>>   GPG Key ID: 0381878A
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
>> Comment: GPGTools - http://gpgtools.org
>> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>>
>> iQEcBAEBAgAGBQJR4TxUAAoJENnmXt7Lmc3KnuEH/0JzCAiG9oMdtzp7KMejAkJh
>> FbtJBVCq+vIHSs9LMmJPTgRkekZBG+DhobMp9sHDvr01Hh8fAFbKvuQAS0Qu5T8t
>> ukxfoF2Ia0C9VPJttiZrMEtKpkAk9vRNpYGad9DdJWrrfh4VcX9zwXp2x32CBJlC
>> ehXyjE4UqN8SfHtBY1XP1WR3YqitplN9HjR7FW48rxHcraNleFtn442mH/2cXJZ1
>> +IllY7MoJf/VdoXWKCZzQhEQjdNek4bTxt0K9cdDg/TwD4yucm9XMO+lw1pSluBX
>> 4Vkh1XCFlQsgWsG5Vf5pQPJdTx0qfOj3mRspz2v/UocnM0iv9aAA5iNEkn5Ouu4=
>> =qMCp
>> -----END PGP SIGNATURE-----
>> _______________________________________________
>> Lognorm mailing list
>> Lognorm at lists.adiscon.com
>> http://lists.adiscon.net/mailman/listinfo/lognorm
>>
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>Lognorm mailing list
>Lognorm at lists.adiscon.com
>http://lists.adiscon.net/mailman/listinfo/lognorm

-- 
Quadrant Information Security
Champ Clark III
o: 904.296.9100 x 101
c: 850.443.2440
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20130713/8182d1f3/attachment.htm>

From cclark at quadrantsec.com  Sat Jul 13 19:12:44 2013
From: cclark at quadrantsec.com (Champ Clark III)
Date: Sat, 13 Jul 2013 13:12:44 -0400
Subject: [Lognorm] Memory Leak in liblognorm?
In-Reply-To: <CADk+mPAeLW9rTeLOjheK47wgWCZHF7AzKbyhL-bdcrC8B_Xa6w@mail.gmail.com>
References: <51E06051.5040901@quadrantsec.com>
	<CADk+mPCMKNr+Be7LedbvvdCYnY=L432TMLcTav+wcJpUb+nfKA@mail.gmail.com>
	<51E06187.2010908@quadrantsec.com>
	<CADk+mPBULO7gdETi-Wo3RgPFXBzwyvTNBE_E_yOVdAbtj_qF3A@mail.gmail.com>
	<51E066BE.2010804@jpserver.net> <51E13C55.5000401@quadrantsec.com>
	<CADk+mPDYAQ_4KvU26W9FTogp4NAW8vbYbWguXrpi7yzxAW3pjw@mail.gmail.com>
	<CADk+mPAeLW9rTeLOjheK47wgWCZHF7AzKbyhL-bdcrC8B_Xa6w@mail.gmail.com>
Message-ID: <51E18A8C.1070502@quadrantsec.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello Rainer,

I got my Ubuntu test env up and configured.  I'm letting it run for a
good bit,  because it seems the leak doesn't always immediately show
up.   I'm running valgrind with the following options:


valgrind --tool=memcheck --leak-check=yes

and

valgrind --tool=memcheck --leak-check=full --show-reachable=yes
- --error-limit=no

Any other flags you want/need?


- -- 
- - Champ Clark III (cclark at quadrantsec.com)
  Quadrant Information Security (http://quadrantsec.com)
  Key Fingerprint: 2E56 C2EB 1B25 C517 D5BA 2DCF 5E70 B2F8 0381 878A
  GPG Key ID: 0381878A
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJR4YqLAAoJENnmXt7Lmc3Km4MIAJVkYU7RstEDtz9YhIZdxsRv
isyjL/W+620CfdDGmvSC7+HX5XEEjbFBW5a2CtNnXi/mAbv++PVszJHVyCK8NfVi
YHCfnqfnAFI8lgtz/18UmUeNIdSwdtkcQUZwof/C1OrCQRkLDADq1tikCOMJ6I7M
Y7CYW2jhFM7I2e35VTsLmlIkMNhglsKhVm50GhJznPfWoKAtajGCe7aVUUW57EMt
5TX+PWGEnNzW7sX97JGWR3dFapvcDgDwEZ+WWE5ZZF2YJcIyulPVyUR0smgtRyLD
/Z9j0QdGEjTuIyeekA0w5+RE0jGxvpX9gB4qK7YJm4b4ptj56Za7tH529Lv+YzI=
=tQhh
-----END PGP SIGNATURE-----

From rgerhards at hq.adiscon.com  Sat Jul 13 20:13:43 2013
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Sat, 13 Jul 2013 20:13:43 +0200
Subject: [Lognorm] Memory Leak in liblognorm?
In-Reply-To: <51E18A8C.1070502@quadrantsec.com>
References: <51E06051.5040901@quadrantsec.com>
	<CADk+mPCMKNr+Be7LedbvvdCYnY=L432TMLcTav+wcJpUb+nfKA@mail.gmail.com>
	<51E06187.2010908@quadrantsec.com>
	<CADk+mPBULO7gdETi-Wo3RgPFXBzwyvTNBE_E_yOVdAbtj_qF3A@mail.gmail.com>
	<51E066BE.2010804@jpserver.net> <51E13C55.5000401@quadrantsec.com>
	<CADk+mPDYAQ_4KvU26W9FTogp4NAW8vbYbWguXrpi7yzxAW3pjw@mail.gmail.com>
	<CADk+mPAeLW9rTeLOjheK47wgWCZHF7AzKbyhL-bdcrC8B_Xa6w@mail.gmail.com>
	<51E18A8C.1070502@quadrantsec.com>
Message-ID: <CADk+mPCCiqrZ0w2ML4PTTa9wOiz3Dg-X6dCffaVO0xYriW-epA@mail.gmail.com>

Sounds good. I am interested in the memleak stats that are printed out at
termination.

Sent from phone, thus brief.
Am 13.07.2013 19:12 schrieb "Champ Clark III" <cclark at quadrantsec.com>:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Hello Rainer,
>
> I got my Ubuntu test env up and configured.  I'm letting it run for a
> good bit,  because it seems the leak doesn't always immediately show
> up.   I'm running valgrind with the following options:
>
>
> valgrind --tool=memcheck --leak-check=yes
>
> and
>
> valgrind --tool=memcheck --leak-check=full --show-reachable=yes
> - --error-limit=no
>
> Any other flags you want/need?
>
>
> - --
> - - Champ Clark III (cclark at quadrantsec.com)
>   Quadrant Information Security (http://quadrantsec.com)
>   Key Fingerprint: 2E56 C2EB 1B25 C517 D5BA 2DCF 5E70 B2F8 0381 878A
>   GPG Key ID: 0381878A
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQEcBAEBAgAGBQJR4YqLAAoJENnmXt7Lmc3Km4MIAJVkYU7RstEDtz9YhIZdxsRv
> isyjL/W+620CfdDGmvSC7+HX5XEEjbFBW5a2CtNnXi/mAbv++PVszJHVyCK8NfVi
> YHCfnqfnAFI8lgtz/18UmUeNIdSwdtkcQUZwof/C1OrCQRkLDADq1tikCOMJ6I7M
> Y7CYW2jhFM7I2e35VTsLmlIkMNhglsKhVm50GhJznPfWoKAtajGCe7aVUUW57EMt
> 5TX+PWGEnNzW7sX97JGWR3dFapvcDgDwEZ+WWE5ZZF2YJcIyulPVyUR0smgtRyLD
> /Z9j0QdGEjTuIyeekA0w5+RE0jGxvpX9gB4qK7YJm4b4ptj56Za7tH529Lv+YzI=
> =tQhh
> -----END PGP SIGNATURE-----
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20130713/2d822a60/attachment.htm>

From cclark at quadrantsec.com  Sat Jul 13 23:24:02 2013
From: cclark at quadrantsec.com (Champ Clark III)
Date: Sat, 13 Jul 2013 17:24:02 -0400
Subject: [Lognorm] Memory Leak in liblognorm?
In-Reply-To: <CADk+mPCCiqrZ0w2ML4PTTa9wOiz3Dg-X6dCffaVO0xYriW-epA@mail.gmail.com>
References: <51E06051.5040901@quadrantsec.com>
	<CADk+mPCMKNr+Be7LedbvvdCYnY=L432TMLcTav+wcJpUb+nfKA@mail.gmail.com>
	<51E06187.2010908@quadrantsec.com>
	<CADk+mPBULO7gdETi-Wo3RgPFXBzwyvTNBE_E_yOVdAbtj_qF3A@mail.gmail.com>
	<51E066BE.2010804@jpserver.net> <51E13C55.5000401@quadrantsec.com>
	<CADk+mPDYAQ_4KvU26W9FTogp4NAW8vbYbWguXrpi7yzxAW3pjw@mail.gmail.com>
	<CADk+mPAeLW9rTeLOjheK47wgWCZHF7AzKbyhL-bdcrC8B_Xa6w@mail.gmail.com>
	<51E18A8C.1070502@quadrantsec.com>
	<CADk+mPCCiqrZ0w2ML4PTTa9wOiz3Dg-X6dCffaVO0xYriW-epA@mail.gmail.com>
Message-ID: <51E1C572.9080003@quadrantsec.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Okay - this is from a short (45 minute) run..

I'll send another in a bit...

tell me if it sheds any light.


On 7/13/13 2:13 PM, Rainer Gerhards wrote:
> Sounds good. I am interested in the memleak stats that are printed
> out at termination.
> 
> Sent from phone, thus brief.
> 
> Am 13.07.2013 19:12 schrieb "Champ Clark III"
> <cclark at quadrantsec.com <mailto:cclark at quadrantsec.com>>:
> 
> Hello Rainer,
> 
> I got my Ubuntu test env up and configured.  I'm letting it run for
> a good bit,  because it seems the leak doesn't always immediately
> show up.   I'm running valgrind with the following options:
> 
> 
> valgrind --tool=memcheck --leak-check=yes
> 
> and
> 
> valgrind --tool=memcheck --leak-check=full --show-reachable=yes 
> --error-limit=no
> 
> Any other flags you want/need?
> 
> 
> _______________________________________________ Lognorm mailing
> list Lognorm at lists.adiscon.com <mailto:Lognorm at lists.adiscon.com> 
> http://lists.adiscon.net/mailman/listinfo/lognorm
> 
> 
> 
> _______________________________________________ Lognorm mailing
> list Lognorm at lists.adiscon.com 
> http://lists.adiscon.net/mailman/listinfo/lognorm
> 

- -- 
- - Champ Clark III (cclark at quadrantsec.com)
  Quadrant Information Security (http://quadrantsec.com)
  Key Fingerprint: 2E56 C2EB 1B25 C517 D5BA 2DCF 5E70 B2F8 0381 878A
  GPG Key ID: 0381878A
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJR4cVyAAoJENnmXt7Lmc3KmTkH/1nKckxxbBcvwxcCM8VCLcJx
fPvtGm2MFToC3KVocfQj+AWiqsnF8ZeC7mvz3j0R85VtTzTm+XVpbWtVPNSXR9Sn
2CNKLZBvJ6VPBw88XmDdG+jcUiNXLka0IE3EfDmf/0BmUeLpGv4UbJla8CR0Xv5g
qJKw/+SQfpaiCMWwi2bUQFiNWnCopZgIglypsqhviF0zOOQJ8dGNCrgPfd3Y6ik6
8b/jCiF2IC+2FehQI7NpXv7HFp6GLeXdGKE6bCBsMkSQccl5sW+akj7buCh6e9S0
Lsj3Ib9qOoWrLs8IeMr4Tuqxdc+M/yOXMaEwK9wRwbz/lUacU/XmvKkNHaRc3Lc=
=WnRt
-----END PGP SIGNATURE-----
-------------- next part --------------
==8756== Memcheck, a memory error detector
==8756== Copyright (C) 2002-2011, and GNU GPL'd, by Julian Seward et al.
==8756== Using Valgrind-3.7.0 and LibVEX; rerun with -h for copyright info
==8756== Command: ./sagan
==8756== 
==8756== Invalid read of size 1
==8756==    at 0x40A995: strlcpy (sagan-strlcpy.c:51)
==8756==    by 0x405A60: Load_Config (sagan-config.c:605)
==8756==    by 0x402AD3: main (sagan.c:330)
==8756==  Address 0x7feffcb60 is just below the stack ptr.  To suppress, use: --workaround-gcc296-bugs=yes
==8756== 
==8756== Invalid read of size 1
==8756==    at 0x40A995: strlcpy (sagan-strlcpy.c:51)
==8756==    by 0x405E0E: Load_Config (sagan-config.c:230)
==8756==    by 0x402AD3: main (sagan.c:330)
==8756==  Address 0x7feffcb60 is just below the stack ptr.  To suppress, use: --workaround-gcc296-bugs=yes
==8756== 
==8756== Conditional jump or move depends on uninitialised value(s)
==8756==    at 0x40AD16: Remove_Spaces (sagan-util.c:155)
==8756==    by 0x405DA5: Load_Config (sagan-config.c:227)
==8756==    by 0x402AD3: main (sagan.c:330)
==8756== 
==8756== Thread 10:
==8756== Conditional jump or move depends on uninitialised value(s)
==8756==    at 0x5DFC3B1: vfprintf (vfprintf.c:1630)
==8756==    by 0x5E24441: vsnprintf (vsnprintf.c:120)
==8756==    by 0x5E04971: snprintf (snprintf.c:35)
==8756==    by 0x41444B: Sagan_Blacklist (stdio2.h:65)
==8756==    by 0x40BD37: Sagan_Processor (sagan-processor.c:123)
==8756==    by 0x595EE99: start_thread (pthread_create.c:308)
==8756== 
==8756== Conditional jump or move depends on uninitialised value(s)
==8756==    at 0x5EB3BB9: inet_pton4 (inet_pton.c:93)
==8756==    by 0x5EB3CDA: inet_pton (inet_pton.c:59)
==8756==    by 0x40B149: IP2Bit (sagan-util.c:253)
==8756==    by 0x414466: Sagan_Blacklist (sagan-blacklist.c:214)
==8756==    by 0x40BD37: Sagan_Processor (sagan-processor.c:123)
==8756==    by 0x595EE99: start_thread (pthread_create.c:308)
==8756== 
==8756== Conditional jump or move depends on uninitialised value(s)
==8756==    at 0x5EB3BC3: inet_pton4 (inet_pton.c:95)
==8756==    by 0x5EB3CDA: inet_pton (inet_pton.c:59)
==8756==    by 0x40B149: IP2Bit (sagan-util.c:253)
==8756==    by 0x414466: Sagan_Blacklist (sagan-blacklist.c:214)
==8756==    by 0x40BD37: Sagan_Processor (sagan-processor.c:123)
==8756==    by 0x595EE99: start_thread (pthread_create.c:308)
==8756== 
==8756== Conditional jump or move depends on uninitialised value(s)
==8756==    at 0x5EB3BE4: inet_pton4 (inet_pton.c:100)
==8756==    by 0x5EB3CDA: inet_pton (inet_pton.c:59)
==8756==    by 0x40B149: IP2Bit (sagan-util.c:253)
==8756==    by 0x414466: Sagan_Blacklist (sagan-blacklist.c:214)
==8756==    by 0x40BD37: Sagan_Processor (sagan-processor.c:123)
==8756==    by 0x595EE99: start_thread (pthread_create.c:308)
==8756== 
==8756== Conditional jump or move depends on uninitialised value(s)
==8756==    at 0x5EB3C01: inet_pton4 (inet_pton.c:93)
==8756==    by 0x5EB3CDA: inet_pton (inet_pton.c:59)
==8756==    by 0x40B149: IP2Bit (sagan-util.c:253)
==8756==    by 0x414466: Sagan_Blacklist (sagan-blacklist.c:214)
==8756==    by 0x40BD37: Sagan_Processor (sagan-processor.c:123)
==8756==    by 0x595EE99: start_thread (pthread_create.c:308)
==8756== 
==8756== Conditional jump or move depends on uninitialised value(s)
==8756==    at 0x5EB3BD0: inet_pton4 (inet_pton.c:98)
==8756==    by 0x5EB3CDA: inet_pton (inet_pton.c:59)
==8756==    by 0x40B149: IP2Bit (sagan-util.c:253)
==8756==    by 0x414466: Sagan_Blacklist (sagan-blacklist.c:214)
==8756==    by 0x40BD37: Sagan_Processor (sagan-processor.c:123)
==8756==    by 0x595EE99: start_thread (pthread_create.c:308)
==8756== 
==8756== Conditional jump or move depends on uninitialised value(s)
==8756==    at 0x5EB3C1B: inet_pton4 (inet_pton.c:108)
==8756==    by 0x5EB3CDA: inet_pton (inet_pton.c:59)
==8756==    by 0x40B149: IP2Bit (sagan-util.c:253)
==8756==    by 0x414466: Sagan_Blacklist (sagan-blacklist.c:214)
==8756==    by 0x40BD37: Sagan_Processor (sagan-processor.c:123)
==8756==    by 0x595EE99: start_thread (pthread_create.c:308)
==8756== 
==8756== Conditional jump or move depends on uninitialised value(s)
==8756==    at 0x414528: Sagan_Blacklist (sagan-blacklist.c:220)
==8756==    by 0x40BD37: Sagan_Processor (sagan-processor.c:123)
==8756==    by 0x595EE99: start_thread (pthread_create.c:308)
==8756== 
==8756== Conditional jump or move depends on uninitialised value(s)
==8756==    at 0x41448B: Sagan_Blacklist (sagan-blacklist.c:220)
==8756==    by 0x40BD37: Sagan_Processor (sagan-processor.c:123)
==8756==    by 0x595EE99: start_thread (pthread_create.c:308)
==8756== 
==8756== Conditional jump or move depends on uninitialised value(s)
==8756==    at 0x414530: Sagan_Blacklist (sagan-blacklist.c:220)
==8756==    by 0x40BD37: Sagan_Processor (sagan-processor.c:123)
==8756==    by 0x595EE99: start_thread (pthread_create.c:308)
==8756== 
==8756== Thread 26:
==8756== Invalid read of size 1
==8756==    at 0x5DFC3B1: vfprintf (vfprintf.c:1630)
==8756==    by 0x5E24441: vsnprintf (vsnprintf.c:120)
==8756==    by 0x5E04971: snprintf (snprintf.c:35)
==8756==    by 0x41444B: Sagan_Blacklist (stdio2.h:65)
==8756==    by 0x40BD37: Sagan_Processor (sagan-processor.c:123)
==8756==    by 0x595EE99: start_thread (pthread_create.c:308)
==8756==  Address 0x149c14ee is not stack'd, malloc'd or (recently) free'd
==8756== 
==8756== Invalid read of size 1
==8756==    at 0x5E2D108: _IO_default_xsputn (genops.c:480)
==8756==    by 0x5DFBF81: vfprintf (vfprintf.c:1630)
==8756==    by 0x5E24441: vsnprintf (vsnprintf.c:120)
==8756==    by 0x5E04971: snprintf (snprintf.c:35)
==8756==    by 0x41444B: Sagan_Blacklist (stdio2.h:65)
==8756==    by 0x40BD37: Sagan_Processor (sagan-processor.c:123)
==8756==    by 0x595EE99: start_thread (pthread_create.c:308)
==8756==  Address 0x149c14ee is not stack'd, malloc'd or (recently) free'd
==8756== 
==8756== Invalid read of size 1
==8756==    at 0x5E2D117: _IO_default_xsputn (genops.c:479)
==8756==    by 0x5DFBF81: vfprintf (vfprintf.c:1630)
==8756==    by 0x5E24441: vsnprintf (vsnprintf.c:120)
==8756==    by 0x5E04971: snprintf (snprintf.c:35)
==8756==    by 0x41444B: Sagan_Blacklist (stdio2.h:65)
==8756==    by 0x40BD37: Sagan_Processor (sagan-processor.c:123)
==8756==    by 0x595EE99: start_thread (pthread_create.c:308)
==8756==  Address 0x149c14f0 is not stack'd, malloc'd or (recently) free'd
==8756== 
==8756== Thread 37:
==8756== Conditional jump or move depends on uninitialised value(s)
==8756==    at 0x40A9A4: strlcpy (sagan-strlcpy.c:51)
==8756==    by 0x41460E: Sagan_Blacklist (sagan-blacklist.c:228)
==8756==    by 0x40BD37: Sagan_Processor (sagan-processor.c:123)
==8756==    by 0x595EE99: start_thread (pthread_create.c:308)
==8756== 
==8756== Conditional jump or move depends on uninitialised value(s)
==8756==    at 0x40A9A4: strlcpy (sagan-strlcpy.c:51)
==8756==    by 0x4145E0: Sagan_Blacklist (sagan-blacklist.c:234)
==8756==    by 0x40BD37: Sagan_Processor (sagan-processor.c:123)
==8756==    by 0x595EE99: start_thread (pthread_create.c:308)
==8756== 
==8756== Conditional jump or move depends on uninitialised value(s)
==8756==    at 0x5DFC3B1: vfprintf (vfprintf.c:1630)
==8756==    by 0x5EBA0DA: __fprintf_chk (fprintf_chk.c:37)
==8756==    by 0x40DA73: Sagan_Alert_File (stdio2.h:98)
==8756==    by 0x40B920: Sagan_Output (sagan-output.c:52)
==8756==    by 0x40C86D: Sagan_Send_Alert (sagan-send-alert.c:55)
==8756==    by 0x414501: Sagan_Blacklist (sagan-blacklist.c:240)
==8756==    by 0x40BD37: Sagan_Processor (sagan-processor.c:123)
==8756==    by 0x595EE99: start_thread (pthread_create.c:308)
==8756== 
==8756== Syscall param write(buf) points to uninitialised byte(s)
==8756==    at 0x5E9792D: ??? (syscall-template.S:82)
==8756==    by 0x5E2A882: _IO_file_write@@GLIBC_2.2.5 (fileops.c:1289)
==8756==    by 0x5E2A749: new_do_write (fileops.c:543)
==8756==    by 0x5E2BEB4: _IO_do_write@@GLIBC_2.2.5 (fileops.c:516)
==8756==    by 0x5E2ADBF: _IO_file_sync@@GLIBC_2.2.5 (fileops.c:918)
==8756==    by 0x5E1FCFA: fflush (iofflush.c:43)
==8756==    by 0x40DAB3: Sagan_Alert_File (sagan-alert.c:66)
==8756==    by 0x40B920: Sagan_Output (sagan-output.c:52)
==8756==    by 0x40C86D: Sagan_Send_Alert (sagan-send-alert.c:55)
==8756==    by 0x414501: Sagan_Blacklist (sagan-blacklist.c:240)
==8756==    by 0x40BD37: Sagan_Processor (sagan-processor.c:123)
==8756==    by 0x595EE99: start_thread (pthread_create.c:308)
==8756==  Address 0x402607f is not stack'd, malloc'd or (recently) free'd
==8756== 
==8756== 
==8756== More than 10000000 total errors detected.  I'm not reporting any more.
==8756== Final error counts will be inaccurate.  Go fix your program!
==8756== Rerun with --error-limit=no to disable this cutoff.  Note
==8756== that errors may occur in your program without prior warning from
==8756== Valgrind, because errors are no longer being displayed.
==8756== 
==8756== 
==8756== HEAP SUMMARY:
==8756==     in use at exit: 10,647,639 bytes in 313,567 blocks
==8756==   total heap usage: 1,223,383 allocs, 909,816 frees, 496,720,983 bytes allocated
==8756== 
==8756== Thread 1:
==8756== 40 bytes in 1 blocks are definitely lost in loss record 28 of 60
==8756==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==8756==    by 0x4123FD: Sagan_Engine_Init (sagan-engine.c:85)
==8756==    by 0x402AD8: main (sagan.c:331)
==8756== 
==8756== 272 bytes in 1 blocks are possibly lost in loss record 34 of 60
==8756==    at 0x4C29DB4: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==8756==    by 0x4012074: _dl_allocate_tls (dl-tls.c:297)
==8756==    by 0x595FABC: pthread_create@@GLIBC_2.2.5 (allocatestack.c:571)
==8756==    by 0x402D1E: main (sagan.c:505)
==8756== 
==8756== 272 bytes in 1 blocks are possibly lost in loss record 35 of 60
==8756==    at 0x4C29DB4: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==8756==    by 0x4012074: _dl_allocate_tls (dl-tls.c:297)
==8756==    by 0x595FABC: pthread_create@@GLIBC_2.2.5 (allocatestack.c:571)
==8756==    by 0x4036F4: main (sagan.c:517)
==8756== 
==8756== 300 (60 direct, 240 indirect) bytes in 1 blocks are definitely lost in loss record 36 of 60
==8756==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==8756==    by 0x5EB65A4: nss_parse_service_list (nsswitch.c:678)
==8756==    by 0x5EB7065: __nss_database_lookup (nsswitch.c:175)
==8756==    by 0x65742A4: ???
==8756==    by 0x5E6F9BC: getpwnam_r@@GLIBC_2.2.5 (getXXbyYY_r.c:256)
==8756==    by 0x5E6F383: getpwnam (getXXbyYY.c:117)
==8756==    by 0x40AF8D: sagan_droppriv (sagan-util.c:92)
==8756==    by 0x402B7E: main (sagan.c:383)
==8756== 
==8756== 348 (60 direct, 288 indirect) bytes in 1 blocks are definitely lost in loss record 37 of 60
==8756==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==8756==    by 0x5EB65A4: nss_parse_service_list (nsswitch.c:678)
==8756==    by 0x5EB7065: __nss_database_lookup (nsswitch.c:175)
==8756==    by 0x6576AC5: ???
==8756==    by 0x5E6D47C: internal_getgrouplist (initgroups.c:114)
==8756==    by 0x5E6D788: initgroups (initgroups.c:221)
==8756==    by 0x40B007: sagan_droppriv (sagan-util.c:104)
==8756==    by 0x402B7E: main (sagan.c:383)
==8756== 
==8756== 13,600 bytes in 50 blocks are possibly lost in loss record 48 of 60
==8756==    at 0x4C29DB4: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==8756==    by 0x4012074: _dl_allocate_tls (dl-tls.c:297)
==8756==    by 0x595FABC: pthread_create@@GLIBC_2.2.5 (allocatestack.c:571)
==8756==    by 0x402D86: main (sagan.c:537)
==8756== 
==8756== 521,360 bytes in 49 blocks are possibly lost in loss record 53 of 60
==8756==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==8756==    by 0x40B997: Sagan_Processor (sagan-processor.c:59)
==8756==    by 0x595EE99: start_thread (pthread_create.c:308)
==8756== 
==8756== 835,776 bytes in 52,236 blocks are definitely lost in loss record 55 of 60
==8756==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==8756==    by 0x5457CD9: es_newStr (string.c:105)
==8756==    by 0x5457D5E: es_newStrFromBuf (string.c:139)
==8756==    by 0x40C318: sagan_normalize_liblognorm (sagan-liblognorm.c:125)
==8756==    by 0x41464F: Sagan_Blacklist (sagan-blacklist.c:167)
==8756==    by 0x40BD37: Sagan_Processor (sagan-processor.c:123)
==8756==    by 0x595EE99: start_thread (pthread_create.c:308)
==8756== 
==8756== 835,776 bytes in 52,236 blocks are definitely lost in loss record 56 of 60
==8756==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==8756==    by 0x5457CD9: es_newStr (string.c:105)
==8756==    by 0x5457D5E: es_newStrFromBuf (string.c:139)
==8756==    by 0x40C35C: sagan_normalize_liblognorm (sagan-liblognorm.c:131)
==8756==    by 0x41464F: Sagan_Blacklist (sagan-blacklist.c:167)
==8756==    by 0x40BD37: Sagan_Processor (sagan-processor.c:123)
==8756==    by 0x595EE99: start_thread (pthread_create.c:308)
==8756== 
==8756== 835,776 bytes in 52,236 blocks are definitely lost in loss record 57 of 60
==8756==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==8756==    by 0x5457CD9: es_newStr (string.c:105)
==8756==    by 0x5457D5E: es_newStrFromBuf (string.c:139)
==8756==    by 0x40C3A0: sagan_normalize_liblognorm (sagan-liblognorm.c:137)
==8756==    by 0x41464F: Sagan_Blacklist (sagan-blacklist.c:167)
==8756==    by 0x40BD37: Sagan_Processor (sagan-processor.c:123)
==8756==    by 0x595EE99: start_thread (pthread_create.c:308)
==8756== 
==8756== 835,776 bytes in 52,236 blocks are definitely lost in loss record 58 of 60
==8756==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==8756==    by 0x5457CD9: es_newStr (string.c:105)
==8756==    by 0x5457D5E: es_newStrFromBuf (string.c:139)
==8756==    by 0x40C3F5: sagan_normalize_liblognorm (sagan-liblognorm.c:144)
==8756==    by 0x41464F: Sagan_Blacklist (sagan-blacklist.c:167)
==8756==    by 0x40BD37: Sagan_Processor (sagan-processor.c:123)
==8756==    by 0x595EE99: start_thread (pthread_create.c:308)
==8756== 
==8756== 835,776 bytes in 52,236 blocks are definitely lost in loss record 59 of 60
==8756==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==8756==    by 0x5457CD9: es_newStr (string.c:105)
==8756==    by 0x5457D5E: es_newStrFromBuf (string.c:139)
==8756==    by 0x40C44A: sagan_normalize_liblognorm (sagan-liblognorm.c:165)
==8756==    by 0x41464F: Sagan_Blacklist (sagan-blacklist.c:167)
==8756==    by 0x40BD37: Sagan_Processor (sagan-processor.c:123)
==8756==    by 0x595EE99: start_thread (pthread_create.c:308)
==8756== 
==8756== 5,191,136 bytes in 52,236 blocks are definitely lost in loss record 60 of 60
==8756==    at 0x4C2B6CD: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==8756==    by 0x5457CD9: es_newStr (string.c:105)
==8756==    by 0x5457D0E: es_newStrFromCStr (string.c:125)
==8756==    by 0x40C2B2: sagan_normalize_liblognorm (sagan-liblognorm.c:114)
==8756==    by 0x41464F: Sagan_Blacklist (sagan-blacklist.c:167)
==8756==    by 0x40BD37: Sagan_Processor (sagan-processor.c:123)
==8756==    by 0x595EE99: start_thread (pthread_create.c:308)
==8756== 
==8756== LEAK SUMMARY:
==8756==    definitely lost: 9,370,176 bytes in 313,419 blocks
==8756==    indirectly lost: 528 bytes in 22 blocks
==8756==      possibly lost: 535,504 bytes in 101 blocks
==8756==    still reachable: 741,431 bytes in 25 blocks
==8756==         suppressed: 0 bytes in 0 blocks
==8756== Reachable blocks (those to which a pointer was found) are not shown.
==8756== To see them, rerun with: --leak-check=full --show-reachable=yes
==8756== 
==8756== For counts of detected and suppressed errors, rerun with: -v
==8756== Use --track-origins=yes to see where uninitialised values come from
==8756== ERROR SUMMARY: 10000013 errors from 33 contexts (suppressed: 2 from 2)
==8756== could not unlink /tmp/vgdb-pipe-from-vgdb-to-8756-by-root-on-???
==8756== could not unlink /tmp/vgdb-pipe-to-vgdb-from-8756-by-root-on-???
==8756== could not unlink /tmp/vgdb-pipe-shared-mem-vgdb-8756-by-root-on-???

From cclark at quadrantsec.com  Sat Jul 13 23:32:58 2013
From: cclark at quadrantsec.com (Champ Clark III)
Date: Sat, 13 Jul 2013 17:32:58 -0400
Subject: [Lognorm] Memory Leak in liblognorm?
In-Reply-To: <51E1C572.9080003@quadrantsec.com>
References: <51E06051.5040901@quadrantsec.com>
	<CADk+mPCMKNr+Be7LedbvvdCYnY=L432TMLcTav+wcJpUb+nfKA@mail.gmail.com>
	<51E06187.2010908@quadrantsec.com>
	<CADk+mPBULO7gdETi-Wo3RgPFXBzwyvTNBE_E_yOVdAbtj_qF3A@mail.gmail.com>
	<51E066BE.2010804@jpserver.net> <51E13C55.5000401@quadrantsec.com>
	<CADk+mPDYAQ_4KvU26W9FTogp4NAW8vbYbWguXrpi7yzxAW3pjw@mail.gmail.com>
	<CADk+mPAeLW9rTeLOjheK47wgWCZHF7AzKbyhL-bdcrC8B_Xa6w@mail.gmail.com>
	<51E18A8C.1070502@quadrantsec.com>
	<CADk+mPCCiqrZ0w2ML4PTTa9wOiz3Dg-X6dCffaVO0xYriW-epA@mail.gmail.com>
	<51E1C572.9080003@quadrantsec.com>
Message-ID: <0f4179f0-c6fc-49d4-86d1-7c835c1c3496@email.android.com>

Just fyi... I havent even looked at the yet myself.   I have another run going with no error limit btw.

Champ Clark III <cclark at quadrantsec.com> wrote:

>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>
>Okay - this is from a short (45 minute) run..
>
>I'll send another in a bit...
>
>tell me if it sheds any light.
>
>
>On 7/13/13 2:13 PM, Rainer Gerhards wrote:
>> Sounds good. I am interested in the memleak stats that are printed
>> out at termination.
>> 
>> Sent from phone, thus brief.
>> 
>> Am 13.07.2013 19:12 schrieb "Champ Clark III"
>> <cclark at quadrantsec.com <mailto:cclark at quadrantsec.com>>:
>> 
>> Hello Rainer,
>> 
>> I got my Ubuntu test env up and configured.  I'm letting it run for
>> a good bit,  because it seems the leak doesn't always immediately
>> show up.   I'm running valgrind with the following options:
>> 
>> 
>> valgrind --tool=memcheck --leak-check=yes
>> 
>> and
>> 
>> valgrind --tool=memcheck --leak-check=full --show-reachable=yes 
>> --error-limit=no
>> 
>> Any other flags you want/need?
>> 
>> 
>> _______________________________________________ Lognorm mailing
>> list Lognorm at lists.adiscon.com <mailto:Lognorm at lists.adiscon.com> 
>> http://lists.adiscon.net/mailman/listinfo/lognorm
>> 
>> 
>> 
>> _______________________________________________ Lognorm mailing
>> list Lognorm at lists.adiscon.com 
>> http://lists.adiscon.net/mailman/listinfo/lognorm
>> 
>
>- -- 
>- - Champ Clark III (cclark at quadrantsec.com)
>  Quadrant Information Security (http://quadrantsec.com)
>  Key Fingerprint: 2E56 C2EB 1B25 C517 D5BA 2DCF 5E70 B2F8 0381 878A
>  GPG Key ID: 0381878A
>-----BEGIN PGP SIGNATURE-----
>Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
>Comment: GPGTools - http://gpgtools.org
>Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
>iQEcBAEBAgAGBQJR4cVyAAoJENnmXt7Lmc3KmTkH/1nKckxxbBcvwxcCM8VCLcJx
>fPvtGm2MFToC3KVocfQj+AWiqsnF8ZeC7mvz3j0R85VtTzTm+XVpbWtVPNSXR9Sn
>2CNKLZBvJ6VPBw88XmDdG+jcUiNXLka0IE3EfDmf/0BmUeLpGv4UbJla8CR0Xv5g
>qJKw/+SQfpaiCMWwi2bUQFiNWnCopZgIglypsqhviF0zOOQJ8dGNCrgPfd3Y6ik6
>8b/jCiF2IC+2FehQI7NpXv7HFp6GLeXdGKE6bCBsMkSQccl5sW+akj7buCh6e9S0
>Lsj3Ib9qOoWrLs8IeMr4Tuqxdc+M/yOXMaEwK9wRwbz/lUacU/XmvKkNHaRc3Lc=
>=WnRt
>-----END PGP SIGNATURE-----
>
>
>------------------------------------------------------------------------
>
>_______________________________________________
>Lognorm mailing list
>Lognorm at lists.adiscon.com
>http://lists.adiscon.net/mailman/listinfo/lognorm

-- 
Quadrant Information Security
Champ Clark III
o: 904.296.9100 x 101
c: 850.443.2440
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20130713/06441205/attachment.htm>

From rgerhards at hq.adiscon.com  Sat Jul 13 23:35:52 2013
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Sat, 13 Jul 2013 23:35:52 +0200
Subject: [Lognorm] Memory Leak in liblognorm?
In-Reply-To: <0f4179f0-c6fc-49d4-86d1-7c835c1c3496@email.android.com>
References: <51E06051.5040901@quadrantsec.com>
	<CADk+mPCMKNr+Be7LedbvvdCYnY=L432TMLcTav+wcJpUb+nfKA@mail.gmail.com>
	<51E06187.2010908@quadrantsec.com>
	<CADk+mPBULO7gdETi-Wo3RgPFXBzwyvTNBE_E_yOVdAbtj_qF3A@mail.gmail.com>
	<51E066BE.2010804@jpserver.net> <51E13C55.5000401@quadrantsec.com>
	<CADk+mPDYAQ_4KvU26W9FTogp4NAW8vbYbWguXrpi7yzxAW3pjw@mail.gmail.com>
	<CADk+mPAeLW9rTeLOjheK47wgWCZHF7AzKbyhL-bdcrC8B_Xa6w@mail.gmail.com>
	<51E18A8C.1070502@quadrantsec.com>
	<CADk+mPCCiqrZ0w2ML4PTTa9wOiz3Dg-X6dCffaVO0xYriW-epA@mail.gmail.com>
	<51E1C572.9080003@quadrantsec.com>
	<0f4179f0-c6fc-49d4-86d1-7c835c1c3496@email.android.com>
Message-ID: <CADk+mPB3AXgQedc4a2tB4yYrKOOLyaZT6YcON6ASM4MwSyiy+A@mail.gmail.com>

Das

Sent from phone, thus brief.
Am 13.07.2013 23:33 schrieb "Champ Clark III" <cclark at quadrantsec.com>:
>
> Just fyi... I havent even looked at the yet myself.

You should ; ) it tells you precisely where the leak is. Its at least one
estr that's not destructed.  I can't get more in depth on the phone.  Just
look at the stack trace :)

I have another run going with no error limit btw.
>
> Champ Clark III <cclark at quadrantsec.com> wrote:
>>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Okay - this is from a short (45 minute) run..
>>
>> I'll send another in a bit...
>>
>> tell me if it sheds any light.
>>
>>
>> On 7/13/13 2:13 PM, Rainer Gerhards wrote:
>>>
>>> Sounds good. I am interested in the memleak stats that are printed
>>> out at termination.
>>>
>>> Sent from phone, thus brief.
>>>
>>> Am 13.07.2013 19:12 schrieb "Champ Clark III"
>>> <cclark at quadrantsec.com <mailto:cclark at quadrantsec.com>>:
>>>
>>> Hello Rainer,
>>>
>>> I got my Ubuntu test env up and configured.  I'm letting it run for
>>> a good bit,  because it seems the leak doesn't always immediately
>>> show up.   I'm running valgrind with the following options:
>>>
>>> valgrind --tool=memcheck --leak-check=yes
>>>
>>> and
>>>
>>> valgrind --tool=memcheck --leak-check=full --show-reachable=yes
>>> --error-limit=no
>>>
>>> Any other flags you want/need?
>>>
>>>
>>> ________________________________
>>> Lognorm mailing
>>> list Lognorm at lists.adiscon.com <mailto:Lognorm at lists.adiscon.com>
>>> http://lists.adiscon.net/mailman/listinfo/lognorm
>>>
>>>
>>>
>>> ________________________________
>>> Lognorm mailing
>>> list Lognorm at lists.adiscon.com
>>> http://lists.adiscon.net/mailman/listinfo/lognorm
>>
>>
>>
>> - --
>> - - Champ Clark III (cclark at quadrantsec.com)
>> Quadrant Information Security (http://quadrantsec.com)
>> Key Fingerprint: 2E56 C2EB 1B25 C517 D5BA 2DCF 5E70 B2F8 0381 878A
>> GPG Key ID: 0381878A
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG/MacGPG2 v2.0.17 (Darwin!
>>  )
>>
>> Comment: GPGTools - http://gpgtools.org
>> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>>
>> iQEcBAEBAgAGBQJR4cVyAAoJENnmXt7Lmc3KmTkH/1nKckxxbBcvwxcCM8VCLcJx
>> fPvtGm2MFToC3KVocfQj+AWiqsnF8ZeC7mvz3j0R85VtTzTm+XVpbWtVPNSXR9Sn
>> 2CNKLZBvJ6VPBw88XmDdG+jcUiNXLka0IE3EfDmf/0BmUeLpGv4UbJla8CR0Xv5g
>> qJKw/+SQfpaiCMWwi2bUQFiNWnCopZgIglypsqhviF0zOOQJ8dGNCrgPfd3Y6ik6
>> 8b/jCiF2IC+2FehQI7NpXv7HFp6GLeXdGKE6bCBsMkSQccl5sW+akj7buCh6e9S0
>> Lsj3Ib9qOoWrLs8IeMr4Tuqxdc+M/yOXMaEwK9wRwbz/lUacU/XmvKkNHaRc3Lc=
>> =WnRt
>> -----END PGP SIGNATURE-----
>>
>> ________________________________
>>
>> Lognorm mailing list
>> Lognorm at lists.adiscon.com
>> http://lists.adiscon.net/mailman/listinfo/lognorm
>
>
> --
> Quadrant Information Security
> Champ Clark III
> o: 904.296.9100 x 101
> c: 850.443.2440
>
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20130713/49c2d6ba/attachment.htm>

From cclark at quadrantsec.com  Sun Jul 14 01:38:48 2013
From: cclark at quadrantsec.com (Champ Clark III)
Date: Sat, 13 Jul 2013 19:38:48 -0400
Subject: [Lognorm] Memory Leak in liblognorm?
In-Reply-To: <CADk+mPB3AXgQedc4a2tB4yYrKOOLyaZT6YcON6ASM4MwSyiy+A@mail.gmail.com>
References: <51E06051.5040901@quadrantsec.com>
	<CADk+mPCMKNr+Be7LedbvvdCYnY=L432TMLcTav+wcJpUb+nfKA@mail.gmail.com>
	<51E06187.2010908@quadrantsec.com>
	<CADk+mPBULO7gdETi-Wo3RgPFXBzwyvTNBE_E_yOVdAbtj_qF3A@mail.gmail.com>
	<51E066BE.2010804@jpserver.net> <51E13C55.5000401@quadrantsec.com>
	<CADk+mPDYAQ_4KvU26W9FTogp4NAW8vbYbWguXrpi7yzxAW3pjw@mail.gmail.com>
	<CADk+mPAeLW9rTeLOjheK47wgWCZHF7AzKbyhL-bdcrC8B_Xa6w@mail.gmail.com>
	<51E18A8C.1070502@quadrantsec.com>
	<CADk+mPCCiqrZ0w2ML4PTTa9wOiz3Dg-X6dCffaVO0xYriW-epA@mail.gmail.com>
	<51E1C572.9080003@quadrantsec.com>
	<0f4179f0-c6fc-49d4-86d1-7c835c1c3496@email.android.com>
	<CADk+mPB3AXgQedc4a2tB4yYrKOOLyaZT6YcON6ASM4MwSyiy+A@mail.gmail.com>
Message-ID: <51E1E508.8040907@quadrantsec.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


I was with my family going to the beach when I blasted out that
valgrind output.  That's what I meant by not getting a change to look.

I've had a short time to look now,  but man... I'm not seeing it...
Later tonight I'm going to clean this up a little bit and see if it
become more clear.

I do use es_deleteStr() to clean that up,  correct?  Because I don't
see where I'm not.

I appreciate the time Rainer.  I'll look a bit more later and see if I
can't locate it.



On 7/13/13 5:35 PM, Rainer Gerhards wrote:
> Das
> 
> Sent from phone, thus brief. Am 13.07.2013 23:33 schrieb "Champ
> Clark III" <cclark at quadrantsec.com 
> <mailto:cclark at quadrantsec.com>>:
>> 
>> Just fyi... I havent even looked at the yet myself.
> 
> You should ; ) it tells you precisely where the leak is. Its at
> least one estr that's not destructed.  I can't get more in depth on
> the phone.  Just look at the stack trace :)
> 
> I have another run going with no error limit btw.
>> 
>> Champ Clark III <cclark at quadrantsec.com
> <mailto:cclark at quadrantsec.com>> wrote:
>>> 
> Okay - this is from a short (45 minute) run..
> 
> I'll send another in a bit...
> 
> tell me if it sheds any light.
> 
> 
> On 7/13/13 2:13 PM, Rainer Gerhards wrote:
>>>>> 
>>>>> Sounds good. I am interested in the memleak stats that are
>>>>> printed out at termination.
>>>>> 
>>>>> Sent from phone, thus brief.
>>>>> 
>>>>> Am 13.07.2013 19:12 schrieb "Champ Clark III" 
>>>>> <cclark at quadrantsec.com <mailto:cclark at quadrantsec.com>
>> <mailto:cclark at quadrantsec.com
>> <mailto:cclark at quadrantsec.com>>>:
>>>>> 
>>>>> Hello Rainer,
>>>>> 
>>>>> I got my Ubuntu test env up and configured.  I'm letting it
>>>>> run for a good bit,  because it seems the leak doesn't
>>>>> always immediately show up.   I'm running valgrind with the
>>>>> following options:
>>>>> 
>>>>> valgrind --tool=memcheck --leak-check=yes
>>>>> 
>>>>> and
>>>>> 
>>>>> valgrind --tool=memcheck --leak-check=full
>>>>> --show-reachable=yes --error-limit=no
>>>>> 
>>>>> Any other flags you want/need?
>>>>> 
>>>>> 
>>>>> ________________________________ Lognorm mailing list
>>>>> Lognorm at lists.adiscon.com
>>>>> <mailto:Lognorm at lists.adiscon.com>
>> <mailto:Lognorm at lists.adiscon.com
>> <mailto:Lognorm at lists.adiscon.com>>
>>>>> http://lists.adiscon.net/mailman/listinfo/lognorm
>>>>> 
>>>>> 
>>>>> 
>>>>> ________________________________ Lognorm mailing list
>>>>> Lognorm at lists.adiscon.com
>>>>> <mailto:Lognorm at lists.adiscon.com> 
>>>>> http://lists.adiscon.net/mailman/listinfo/lognorm
> 
> 
> 
>>> 
>>> ________________________________
>>> 
>>> Lognorm mailing list Lognorm at lists.adiscon.com
>>> <mailto:Lognorm at lists.adiscon.com> 
>>> http://lists.adiscon.net/mailman/listinfo/lognorm
>> 
>> 
>> -- Quadrant Information Security Champ Clark III o: 904.296.9100
>> x 101 c: 850.443.2440
>> 
>> _______________________________________________ Lognorm mailing
>> list Lognorm at lists.adiscon.com
>> <mailto:Lognorm at lists.adiscon.com> 
>> http://lists.adiscon.net/mailman/listinfo/lognorm
>> 
> 
> 
> 
> _______________________________________________ Lognorm mailing
> list Lognorm at lists.adiscon.com 
> http://lists.adiscon.net/mailman/listinfo/lognorm
> 

- -- 
- - Champ Clark III (cclark at quadrantsec.com)
  Quadrant Information Security (http://quadrantsec.com)
  Key Fingerprint: 2E56 C2EB 1B25 C517 D5BA 2DCF 5E70 B2F8 0381 878A
  GPG Key ID: 0381878A
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJR4eUIAAoJENnmXt7Lmc3KEvUH/3GySdbJ0tvz4fEdK7lizUCS
eyC20YUaPYMvdt7vJe5unpRAGzrczi1GyDgWMoxgKhjX71xuQnqEi8KWXxcx3Boj
Cz8xjmPJUv0IcQWvzqVxw3H2qA0BNwGnyTD6OY5BIRE9RYR0N58iniULdEAU0zQT
T2hhBq30/xKCWhQ54CDmmBpg4gwVHAfJtskNGa43H119jEOt0gisJ8IUT6QkkKQR
jdl5CLgUaIoGW76oN8I86j4toditngHuviDD9nM8MIh0S+s5bFnCAM5lWlQYfo7t
70nSlAD1/bvdCfOfe1OwNMuoTDWFQOSbAHjITo2NuTJT8ggzVpqTf32z/oqTZKA=
=+VQJ
-----END PGP SIGNATURE-----

From rgerhards at hq.adiscon.com  Sun Jul 14 14:48:56 2013
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Sun, 14 Jul 2013 14:48:56 +0200
Subject: [Lognorm] Memory Leak in liblognorm?
In-Reply-To: <51E1E508.8040907@quadrantsec.com>
References: <51E06051.5040901@quadrantsec.com>
	<CADk+mPCMKNr+Be7LedbvvdCYnY=L432TMLcTav+wcJpUb+nfKA@mail.gmail.com>
	<51E06187.2010908@quadrantsec.com>
	<CADk+mPBULO7gdETi-Wo3RgPFXBzwyvTNBE_E_yOVdAbtj_qF3A@mail.gmail.com>
	<51E066BE.2010804@jpserver.net> <51E13C55.5000401@quadrantsec.com>
	<CADk+mPDYAQ_4KvU26W9FTogp4NAW8vbYbWguXrpi7yzxAW3pjw@mail.gmail.com>
	<CADk+mPAeLW9rTeLOjheK47wgWCZHF7AzKbyhL-bdcrC8B_Xa6w@mail.gmail.com>
	<51E18A8C.1070502@quadrantsec.com>
	<CADk+mPCCiqrZ0w2ML4PTTa9wOiz3Dg-X6dCffaVO0xYriW-epA@mail.gmail.com>
	<51E1C572.9080003@quadrantsec.com>
	<0f4179f0-c6fc-49d4-86d1-7c835c1c3496@email.android.com>
	<CADk+mPB3AXgQedc4a2tB4yYrKOOLyaZT6YcON6ASM4MwSyiy+A@mail.gmail.com>
	<51E1E508.8040907@quadrantsec.com>
Message-ID: <CADk+mPD04nm+pHb3h4nH=qgZVojH+s-Xq=xma+kTeTUzLCAYEw@mail.gmail.com>

On Sun, Jul 14, 2013 at 1:38 AM, Champ Clark III <cclark at quadrantsec.com>wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
> I was with my family going to the beach when I blasted out that
> valgrind output.  That's what I meant by not getting a change to look.
>
>
No problem at all, I guess we have similiar constraints.


> I've had a short time to look now,  but man... I'm not seeing it...
> Later tonight I'm going to clean this up a little bit and see if it
> become more clear.
>
>
Well, it may take a short while to get used to reading the output. Take
this here for example:

==8756== 835,776 bytes in 52,236 blocks are definitely lost in loss record
56 of 60
so we lot 52,236 "object", alltogehter 835,776 bytes; now comes the
stacktrace where they were allocated

==8756==    at 0x4C2B6CD: malloc (in
/usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==8756==    by 0x5457CD9: es_newStr (string.c:105)
so its about estr

==8756==    by 0x5457D5E: es_newStrFromBuf (string.c:139)
==8756==    by 0x40C35C: sagan_normalize_liblognorm (sagan-liblognorm.c:131)
This is where your code called it (sagan-liblognorm.c, line 131)

==8756==    by 0x41464F: Sagan_Blacklist (sagan-blacklist.c:167)
==8756==    by 0x40BD37: Sagan_Processor (sagan-processor.c:123)
==8756==    by 0x595EE99: start_thread (pthread_create.c:308



> I do use es_deleteStr() to clean that up,  correct?  Because I don't
> see where I'm not.
>
>
yup - you should now check the code what happens to that object that was
allocated in line 131.


> I appreciate the time Rainer.  I'll look a bit more later and see if I
> can't locate it.
>
>
side-note: there are many off-by-one errors mentioned in the valgrind log.
I highly suggest to work through them, as they may lead to "interesting"
effects. rsyslog stability greatly improved when I made sure there are no
valgrind exception and I highly recommend doing that.

Rainer

>
>
> On 7/13/13 5:35 PM, Rainer Gerhards wrote:
> > Das
> >
> > Sent from phone, thus brief. Am 13.07.2013 23:33 schrieb "Champ
> > Clark III" <cclark at quadrantsec.com
> > <mailto:cclark at quadrantsec.com>>:
> >>
> >> Just fyi... I havent even looked at the yet myself.
> >
> > You should ; ) it tells you precisely where the leak is. Its at
> > least one estr that's not destructed.  I can't get more in depth on
> > the phone.  Just look at the stack trace :)
> >
> > I have another run going with no error limit btw.
> >>
> >> Champ Clark III <cclark at quadrantsec.com
> > <mailto:cclark at quadrantsec.com>> wrote:
> >>>
> > Okay - this is from a short (45 minute) run..
> >
> > I'll send another in a bit...
> >
> > tell me if it sheds any light.
> >
> >
> > On 7/13/13 2:13 PM, Rainer Gerhards wrote:
> >>>>>
> >>>>> Sounds good. I am interested in the memleak stats that are
> >>>>> printed out at termination.
> >>>>>
> >>>>> Sent from phone, thus brief.
> >>>>>
> >>>>> Am 13.07.2013 19:12 schrieb "Champ Clark III"
> >>>>> <cclark at quadrantsec.com <mailto:cclark at quadrantsec.com>
> >> <mailto:cclark at quadrantsec.com
> >> <mailto:cclark at quadrantsec.com>>>:
> >>>>>
> >>>>> Hello Rainer,
> >>>>>
> >>>>> I got my Ubuntu test env up and configured.  I'm letting it
> >>>>> run for a good bit,  because it seems the leak doesn't
> >>>>> always immediately show up.   I'm running valgrind with the
> >>>>> following options:
> >>>>>
> >>>>> valgrind --tool=memcheck --leak-check=yes
> >>>>>
> >>>>> and
> >>>>>
> >>>>> valgrind --tool=memcheck --leak-check=full
> >>>>> --show-reachable=yes --error-limit=no
> >>>>>
> >>>>> Any other flags you want/need?
> >>>>>
> >>>>>
> >>>>> ________________________________ Lognorm mailing list
> >>>>> Lognorm at lists.adiscon.com
> >>>>> <mailto:Lognorm at lists.adiscon.com>
> >> <mailto:Lognorm at lists.adiscon.com
> >> <mailto:Lognorm at lists.adiscon.com>>
> >>>>> http://lists.adiscon.net/mailman/listinfo/lognorm
> >>>>>
> >>>>>
> >>>>>
> >>>>> ________________________________ Lognorm mailing list
> >>>>> Lognorm at lists.adiscon.com
> >>>>> <mailto:Lognorm at lists.adiscon.com>
> >>>>> http://lists.adiscon.net/mailman/listinfo/lognorm
> >
> >
> >
> >>>
> >>> ________________________________
> >>>
> >>> Lognorm mailing list Lognorm at lists.adiscon.com
> >>> <mailto:Lognorm at lists.adiscon.com>
> >>> http://lists.adiscon.net/mailman/listinfo/lognorm
> >>
> >>
> >> -- Quadrant Information Security Champ Clark III o: 904.296.9100
> >> x 101 c: 850.443.2440
> >>
> >> _______________________________________________ Lognorm mailing
> >> list Lognorm at lists.adiscon.com
> >> <mailto:Lognorm at lists.adiscon.com>
> >> http://lists.adiscon.net/mailman/listinfo/lognorm
> >>
> >
> >
> >
> > _______________________________________________ Lognorm mailing
> > list Lognorm at lists.adiscon.com
> > http://lists.adiscon.net/mailman/listinfo/lognorm
> >
>
> - --
> - - Champ Clark III (cclark at quadrantsec.com)
>   Quadrant Information Security (http://quadrantsec.com)
>   Key Fingerprint: 2E56 C2EB 1B25 C517 D5BA 2DCF 5E70 B2F8 0381 878A
>   GPG Key ID: 0381878A
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQEcBAEBAgAGBQJR4eUIAAoJENnmXt7Lmc3KEvUH/3GySdbJ0tvz4fEdK7lizUCS
> eyC20YUaPYMvdt7vJe5unpRAGzrczi1GyDgWMoxgKhjX71xuQnqEi8KWXxcx3Boj
> Cz8xjmPJUv0IcQWvzqVxw3H2qA0BNwGnyTD6OY5BIRE9RYR0N58iniULdEAU0zQT
> T2hhBq30/xKCWhQ54CDmmBpg4gwVHAfJtskNGa43H119jEOt0gisJ8IUT6QkkKQR
> jdl5CLgUaIoGW76oN8I86j4toditngHuviDD9nM8MIh0S+s5bFnCAM5lWlQYfo7t
> 70nSlAD1/bvdCfOfe1OwNMuoTDWFQOSbAHjITo2NuTJT8ggzVpqTf32z/oqTZKA=
> =+VQJ
> -----END PGP SIGNATURE-----
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20130714/e11193c2/attachment-0001.htm>

From rgerhards at hq.adiscon.com  Sun Jul 14 14:52:26 2013
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Sun, 14 Jul 2013 14:52:26 +0200
Subject: [Lognorm] Memory Leak in liblognorm?
In-Reply-To: <CADk+mPD04nm+pHb3h4nH=qgZVojH+s-Xq=xma+kTeTUzLCAYEw@mail.gmail.com>
References: <51E06051.5040901@quadrantsec.com>
	<CADk+mPCMKNr+Be7LedbvvdCYnY=L432TMLcTav+wcJpUb+nfKA@mail.gmail.com>
	<51E06187.2010908@quadrantsec.com>
	<CADk+mPBULO7gdETi-Wo3RgPFXBzwyvTNBE_E_yOVdAbtj_qF3A@mail.gmail.com>
	<51E066BE.2010804@jpserver.net> <51E13C55.5000401@quadrantsec.com>
	<CADk+mPDYAQ_4KvU26W9FTogp4NAW8vbYbWguXrpi7yzxAW3pjw@mail.gmail.com>
	<CADk+mPAeLW9rTeLOjheK47wgWCZHF7AzKbyhL-bdcrC8B_Xa6w@mail.gmail.com>
	<51E18A8C.1070502@quadrantsec.com>
	<CADk+mPCCiqrZ0w2ML4PTTa9wOiz3Dg-X6dCffaVO0xYriW-epA@mail.gmail.com>
	<51E1C572.9080003@quadrantsec.com>
	<0f4179f0-c6fc-49d4-86d1-7c835c1c3496@email.android.com>
	<CADk+mPB3AXgQedc4a2tB4yYrKOOLyaZT6YcON6ASM4MwSyiy+A@mail.gmail.com>
	<51E1E508.8040907@quadrantsec.com>
	<CADk+mPD04nm+pHb3h4nH=qgZVojH+s-Xq=xma+kTeTUzLCAYEw@mail.gmail.com>
Message-ID: <CADk+mPDshYyt4Q_2-Gds3KakAUEx642cacC__Th9JYixLr2DNw@mail.gmail.com>

ah, now it is easy to see (for me ;)). Look at line 125+. You do repeatedly:

 propName = es_newStrFromBuf(...);

but you never delete propName before you assign a new value to it ;)

There may be other problems, I had just focussed on the first valgrind
report. Let me know if this get's you going.

Rainer




On Sun, Jul 14, 2013 at 2:48 PM, Rainer Gerhards
<rgerhards at hq.adiscon.com>wrote:

>
> On Sun, Jul 14, 2013 at 1:38 AM, Champ Clark III <cclark at quadrantsec.com>wrote:
>
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>>
>> I was with my family going to the beach when I blasted out that
>> valgrind output.  That's what I meant by not getting a change to look.
>>
>>
> No problem at all, I guess we have similiar constraints.
>
>
>> I've had a short time to look now,  but man... I'm not seeing it...
>> Later tonight I'm going to clean this up a little bit and see if it
>> become more clear.
>>
>>
> Well, it may take a short while to get used to reading the output. Take
> this here for example:
>
> ==8756== 835,776 bytes in 52,236 blocks are definitely lost in loss record
> 56 of 60
> so we lot 52,236 "object", alltogehter 835,776 bytes; now comes the
> stacktrace where they were allocated
>
> ==8756==    at 0x4C2B6CD: malloc (in
> /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==8756==    by 0x5457CD9: es_newStr (string.c:105)
> so its about estr
>
> ==8756==    by 0x5457D5E: es_newStrFromBuf (string.c:139)
> ==8756==    by 0x40C35C: sagan_normalize_liblognorm
> (sagan-liblognorm.c:131)
> This is where your code called it (sagan-liblognorm.c, line 131)
>
> ==8756==    by 0x41464F: Sagan_Blacklist (sagan-blacklist.c:167)
> ==8756==    by 0x40BD37: Sagan_Processor (sagan-processor.c:123)
> ==8756==    by 0x595EE99: start_thread (pthread_create.c:308
>
>
>
>> I do use es_deleteStr() to clean that up,  correct?  Because I don't
>> see where I'm not.
>>
>>
> yup - you should now check the code what happens to that object that was
> allocated in line 131.
>
>
>> I appreciate the time Rainer.  I'll look a bit more later and see if I
>> can't locate it.
>>
>>
> side-note: there are many off-by-one errors mentioned in the valgrind log.
> I highly suggest to work through them, as they may lead to "interesting"
> effects. rsyslog stability greatly improved when I made sure there are no
> valgrind exception and I highly recommend doing that.
>
> Rainer
>
>>
>>
>> On 7/13/13 5:35 PM, Rainer Gerhards wrote:
>> > Das
>> >
>> > Sent from phone, thus brief. Am 13.07.2013 23:33 schrieb "Champ
>> > Clark III" <cclark at quadrantsec.com
>> > <mailto:cclark at quadrantsec.com>>:
>> >>
>> >> Just fyi... I havent even looked at the yet myself.
>> >
>> > You should ; ) it tells you precisely where the leak is. Its at
>> > least one estr that's not destructed.  I can't get more in depth on
>> > the phone.  Just look at the stack trace :)
>> >
>> > I have another run going with no error limit btw.
>> >>
>> >> Champ Clark III <cclark at quadrantsec.com
>> > <mailto:cclark at quadrantsec.com>> wrote:
>> >>>
>> > Okay - this is from a short (45 minute) run..
>> >
>> > I'll send another in a bit...
>> >
>> > tell me if it sheds any light.
>> >
>> >
>> > On 7/13/13 2:13 PM, Rainer Gerhards wrote:
>> >>>>>
>> >>>>> Sounds good. I am interested in the memleak stats that are
>> >>>>> printed out at termination.
>> >>>>>
>> >>>>> Sent from phone, thus brief.
>> >>>>>
>> >>>>> Am 13.07.2013 19:12 schrieb "Champ Clark III"
>> >>>>> <cclark at quadrantsec.com <mailto:cclark at quadrantsec.com>
>> >> <mailto:cclark at quadrantsec.com
>> >> <mailto:cclark at quadrantsec.com>>>:
>> >>>>>
>> >>>>> Hello Rainer,
>> >>>>>
>> >>>>> I got my Ubuntu test env up and configured.  I'm letting it
>> >>>>> run for a good bit,  because it seems the leak doesn't
>> >>>>> always immediately show up.   I'm running valgrind with the
>> >>>>> following options:
>> >>>>>
>> >>>>> valgrind --tool=memcheck --leak-check=yes
>> >>>>>
>> >>>>> and
>> >>>>>
>> >>>>> valgrind --tool=memcheck --leak-check=full
>> >>>>> --show-reachable=yes --error-limit=no
>> >>>>>
>> >>>>> Any other flags you want/need?
>> >>>>>
>> >>>>>
>> >>>>> ________________________________ Lognorm mailing list
>> >>>>> Lognorm at lists.adiscon.com
>> >>>>> <mailto:Lognorm at lists.adiscon.com>
>> >> <mailto:Lognorm at lists.adiscon.com
>> >> <mailto:Lognorm at lists.adiscon.com>>
>> >>>>> http://lists.adiscon.net/mailman/listinfo/lognorm
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> ________________________________ Lognorm mailing list
>> >>>>> Lognorm at lists.adiscon.com
>> >>>>> <mailto:Lognorm at lists.adiscon.com>
>> >>>>> http://lists.adiscon.net/mailman/listinfo/lognorm
>> >
>> >
>> >
>> >>>
>> >>> ________________________________
>> >>>
>> >>> Lognorm mailing list Lognorm at lists.adiscon.com
>> >>> <mailto:Lognorm at lists.adiscon.com>
>> >>> http://lists.adiscon.net/mailman/listinfo/lognorm
>> >>
>> >>
>> >> -- Quadrant Information Security Champ Clark III o: 904.296.9100
>> >> x 101 c: 850.443.2440
>> >>
>> >> _______________________________________________ Lognorm mailing
>> >> list Lognorm at lists.adiscon.com
>> >> <mailto:Lognorm at lists.adiscon.com>
>> >> http://lists.adiscon.net/mailman/listinfo/lognorm
>> >>
>> >
>> >
>> >
>> > _______________________________________________ Lognorm mailing
>> > list Lognorm at lists.adiscon.com
>> > http://lists.adiscon.net/mailman/listinfo/lognorm
>> >
>>
>> - --
>> - - Champ Clark III (cclark at quadrantsec.com)
>>   Quadrant Information Security (http://quadrantsec.com)
>>   Key Fingerprint: 2E56 C2EB 1B25 C517 D5BA 2DCF 5E70 B2F8 0381 878A
>>   GPG Key ID: 0381878A
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
>> Comment: GPGTools - http://gpgtools.org
>> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>>
>> iQEcBAEBAgAGBQJR4eUIAAoJENnmXt7Lmc3KEvUH/3GySdbJ0tvz4fEdK7lizUCS
>> eyC20YUaPYMvdt7vJe5unpRAGzrczi1GyDgWMoxgKhjX71xuQnqEi8KWXxcx3Boj
>> Cz8xjmPJUv0IcQWvzqVxw3H2qA0BNwGnyTD6OY5BIRE9RYR0N58iniULdEAU0zQT
>> T2hhBq30/xKCWhQ54CDmmBpg4gwVHAfJtskNGa43H119jEOt0gisJ8IUT6QkkKQR
>> jdl5CLgUaIoGW76oN8I86j4toditngHuviDD9nM8MIh0S+s5bFnCAM5lWlQYfo7t
>> 70nSlAD1/bvdCfOfe1OwNMuoTDWFQOSbAHjITo2NuTJT8ggzVpqTf32z/oqTZKA=
>> =+VQJ
>> -----END PGP SIGNATURE-----
>> _______________________________________________
>> Lognorm mailing list
>> Lognorm at lists.adiscon.com
>> http://lists.adiscon.net/mailman/listinfo/lognorm
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20130714/3eca8bdb/attachment.htm>

From rgerhards at hq.adiscon.com  Mon Jul 15 09:32:45 2013
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 15 Jul 2013 09:32:45 +0200
Subject: [Lognorm] log classification with liblognorm
In-Reply-To: <701C784DE99B0C4C9FDA9EDE66E23E661E4CAD930B@HQ-MB-07.ba.ad.ssa.gov>
References: <701C784DE99B0C4C9FDA9EDE66E23E661E4CAD9300@HQ-MB-07.ba.ad.ssa.gov>
	<alpine.DEB.2.02.1306261210020.22572@nftneq.ynat.uz>
	<701C784DE99B0C4C9FDA9EDE66E23E661E4CAD930B@HQ-MB-07.ba.ad.ssa.gov>
Message-ID: <CADk+mPBM3qPHVU91mSh-G175FruVUmT6MaKeHU6iEiz+K2P+DA@mail.gmail.com>

David made me aware of this thread. Sorry for not responding further, I've
been very busy and probably overlooked it. More inline...

On Wed, Jun 26, 2013 at 10:41 PM, Castillo, Jose Contractor <
Jose.Castillo at ssa.gov> wrote:

>  David,
>
> I modified the template:
> template(name="testFormat" type="string" string="%$!all-json%,tag='%$!*
> mytag*%'")
>
> I also modified the rule to include mytag as a tag:
> rule=*mytag*:%date:word% %host:ipv4% :
> %%ASA-%number0:number%-%number1:number%: Denied ICMP type=%number2:number%,
> code=%number3:number% from %origin:ipv4% on interface outside
>
> And now I getting :
> { "origin": "77.2.2.2", "number3": "0", "number2": "8", "number1":
> "313001", "number0": "3", "host": "0.0.0.0", "date":
> "2013-06-26T10:47:42+01:00" },tag=''
>
> No tags are being written to the output file.
>
> What am I doing wrong?
>

I need to check the code, but I think you are doing everything right. IIRC,
tags were meant as classification method for rsyslog's engine (if-stmts). I
think outputting them was not at the table when this was written. I am 90%
sure, and will try to verify the rest within this week.
Rainer

>
> Thanks,
>
> Jose
>
> -----Original Message-----
> From: lognorm-bounces at lists.adiscon.com [
> mailto:lognorm-bounces at lists.adiscon.com<lognorm-bounces at lists.adiscon.com>]
> On Behalf Of David Lang
> Sent: Wednesday, June 26, 2013 3:11 PM
> To: lognorm
> Subject: Re: [Lognorm] log classification with liblognorm
>
> the tags are accessed in rsyslog by $!<tag name>
>
> so where you would have done %hostname% you could do %$!mytag%
>
> David Lang
>
> On Wed, 26 Jun 2013, Castillo, Jose   Contractor wrote:
>
> > Date: Wed, 26 Jun 2013 16:00:06 -0400
> > From: "Castillo, Jose   Contractor" <Jose.Castillo at ssa.gov>
> > Reply-To: lognorm <lognorm at lists.adiscon.com>
> > To: "lognorm at lists.adiscon.com" <lognorm at lists.adiscon.com>
> > Subject: [Lognorm] log classification with liblognorm
> >
> > Hello all,
> >
> > I started to play using rsyslog/liblognorm a week ago, I'm testing them
> on a CentOS 6.4 virtual machine, and next packages have been installed from
> Adiscon repository:
> > rsyslog-mysql-7.4.1-1.el6.x86_64
> > rsyslog-mmjsonparse-7.4.1-1.el6.x86_64
> > rsyslog-7.4.1-1.el6.x86_64
> > rsyslog-mmnormalize-7.4.1-1.el6.x86_64
> > rsyslog-udpspoof-7.4.1-1.el6.x86_64
> > rsyslog-elasticsearch-7.4.1-1.el6.x86_64
> >
> > I was reading next link, and now I'm trying to select subsets of
> messages based on liblognorm tags.
> > http://blog.gerhards.net/2011/04/log-classification-with-liblognorm.ht
> > ml
> >
> > So far, I was able to parse a message using mmnormalize module, and now
> I want to create outputs based on the liblognorm tag(s).
> > Is this option supported within rsyslog.conf? Any example how to do it?
> >
> > Below my testing files:
> >
> > =================================================================
> > rsyslog.conf file:
> > module (load="imudp")
> > module (load="mmnormalize")
> > module (load="mmjsonparse")
> >
> > input(type="imudp" address="192.168.1.1" port="514" ruleset="test")
> >
> > template(name="testFormat" type="string" string="%$!all-json%\n")
> >
> > ruleset(name="test") {
> >        action(type="mmnormalize" userawmsg="on"
> rulebase="/data/syslog/rulebase.rb")
> >        action(type="omfile" file="/data/syslog/test-syslog.log"
> > template="testFormat") }
> >
> > rulebase.rb file:
> > rule=test:%date:word% %host:ipv4% :
> > %%ASA-%number0:number%-%number1:number%: Denied ICMP
> > type=%number2:number%, code=%number3:number% from %origin:ipv4% on
> > interface outside
> >
> > test-syslog.log file:
> > { "origin": "77.2.2.2", "number3": "0", "number2": "8", "number1":
> > "313001", "number0": "3", "host": "0.0.0.0", "date":
> > "2013-06-26T10:47:42+01:00" }
> >
> > Message:
> > "2013-06-26T10:47:42+01:00 0.0.0.0 : %ASA-3-313001: Denied ICMP type=8,
> code=0 from 77.2.2.2 on interface outside"
> > ======================================================================
> >
> >
> > Thanks in advance for any help,
> >
> > Jose Castillo
> >
> >
> >
> >
>
>
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20130715/139dcdde/attachment-0001.htm>

From cclark at quadrantsec.com  Mon Jul 15 15:06:03 2013
From: cclark at quadrantsec.com (Champ Clark III)
Date: Mon, 15 Jul 2013 09:06:03 -0400
Subject: [Lognorm] Memory Leak in liblognorm?
In-Reply-To: <CADk+mPDshYyt4Q_2-Gds3KakAUEx642cacC__Th9JYixLr2DNw@mail.gmail.com>
References: <51E06051.5040901@quadrantsec.com>
	<CADk+mPCMKNr+Be7LedbvvdCYnY=L432TMLcTav+wcJpUb+nfKA@mail.gmail.com>
	<51E06187.2010908@quadrantsec.com>
	<CADk+mPBULO7gdETi-Wo3RgPFXBzwyvTNBE_E_yOVdAbtj_qF3A@mail.gmail.com>
	<51E066BE.2010804@jpserver.net> <51E13C55.5000401@quadrantsec.com>
	<CADk+mPDYAQ_4KvU26W9FTogp4NAW8vbYbWguXrpi7yzxAW3pjw@mail.gmail.com>
	<CADk+mPAeLW9rTeLOjheK47wgWCZHF7AzKbyhL-bdcrC8B_Xa6w@mail.gmail.com>
	<51E18A8C.1070502@quadrantsec.com>
	<CADk+mPCCiqrZ0w2ML4PTTa9wOiz3Dg-X6dCffaVO0xYriW-epA@mail.gmail.com>
	<51E1C572.9080003@quadrantsec.com>
	<0f4179f0-c6fc-49d4-86d1-7c835c1c3496@email.android.com>
	<CADk+mPB3AXgQedc4a2tB4yYrKOOLyaZT6YcON6ASM4MwSyiy+A@mail.gmail.com>
	<51E1E508.8040907@quadrantsec.com>
	<CADk+mPD04nm+pHb3h4nH=qgZVojH+s-Xq=xma+kTeTUzLCAYEw@mail.gmail.com>
	<CADk+mPDshYyt4Q_2-Gds3KakAUEx642cacC__Th9JYixLr2DNw@mail.gmail.com>
Message-ID: <51E3F3BB.7050606@quadrantsec.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ahhh!  I wasn't aware I needed to do that!  I thought I could use it
then at the end call  es_deleteStr(propName) once and be done!

I now call after each usage and valgrind now no longer report the leak
there.  Woo! :)

There is one more report from valgrind that I don't quite get.  It is:

==10919== 323,832 bytes in 3,305 blocks are definitely lost in loss
record 53 of 55
==10919==    at 0x4C2B6CD: malloc (in
/usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==10919==    by 0x5457CD9: es_newStr (string.c:105)
==10919==    by 0x5457D0E: es_newStrFromCStr (string.c:125)
==10919==    by 0x40C2B6: sagan_normalize_liblognorm
(sagan-liblognorm.c:114)
==10919==    by 0x41467F: Sagan_Blacklist (sagan-blacklist.c:167)
==10919==    by 0x40BD37: Sagan_Processor (sagan-processor.c:123)
==10919==    by 0x595EE99: start_thread (pthread_create.c:308)


Here is sagan-liblognorm.c line 114+

- --<snip>--

str = es_newStrFromCStr(syslog_msg, strlen(syslog_msg));  /* Line 114 */
ln_normalize(ctx, str, &lnevent);

if(lnevent != NULL) {

  es_emptyStr(str);
  ee_fmtEventToRFC5424(lnevent, &str);
  cstr = es_str2cstr(str, NULL);
  es_deleteStr(str);             /* Thought this would clean it up! */

- --<snip>--


I agree with you and going through the rest of the valgrind report
with Sagan.  It's something I've been meaning to do for a while,  but
just go around to it.  I should have done it a long time again.

Thank again Mr Rainer! :)




On 7/14/13 8:52 AM, Rainer Gerhards wrote:
> ah, now it is easy to see (for me ;)). Look at line 125+. You do
> repeatedly:
> 
> propName = es_newStrFromBuf(...);
> 
> 
> but you never delete propName before you assign a new value to it
> ;)
> 
> There may be other problems, I had just focussed on the first
> valgrind report. Let me know if this get's you going.
> 
> 

- -- 
- - Champ Clark III (cclark at quadrantsec.com)
  Quadrant Information Security (http://quadrantsec.com)
  Key Fingerprint: 2E56 C2EB 1B25 C517 D5BA 2DCF 5E70 B2F8 0381 878A
  GPG Key ID: 0381878A
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJR4/O7AAoJENnmXt7Lmc3KT+8H/2P0/y5f2u5SQ/xt35mOlr6B
x5dj6qJSlKmoMGhOUc5wGz2YclrKEgBC+kF26qSNIe1YD8oSxRMbJxwSKY9iiCJg
PSNm39sQ5t9yOC5uqp0wjvsaELgeFjvlD96ArrU8dgfD21bNZrCAhBWz5+6ttA+z
Gner7Ic0t9EpgjXTsDPDoFchHm8J/QunQ3mcbqGonpl0GrlgZ0nYzNz8gosQiIV+
TazKVPaKp5pvafH1vdO3JDROEpnRbH++QQspj0+9nlDN0mgCL8YtjwkycFAt7++m
RWtWIfmQnpO+Xwx07a9HS2JmSk53+9wi058JHdTYHIwF16ns+4Y1iRJhegt7Pd4=
=cjvP
-----END PGP SIGNATURE-----

From Jose.Castillo at ssa.gov  Mon Jul 15 16:07:21 2013
From: Jose.Castillo at ssa.gov (Castillo, Jose   Contractor)
Date: Mon, 15 Jul 2013 10:07:21 -0400
Subject: [Lognorm] log classification with liblognorm
In-Reply-To: <CADk+mPBM3qPHVU91mSh-G175FruVUmT6MaKeHU6iEiz+K2P+DA@mail.gmail.com>
References: <701C784DE99B0C4C9FDA9EDE66E23E661E4CAD9300@HQ-MB-07.ba.ad.ssa.gov>
	<alpine.DEB.2.02.1306261210020.22572@nftneq.ynat.uz>
	<701C784DE99B0C4C9FDA9EDE66E23E661E4CAD930B@HQ-MB-07.ba.ad.ssa.gov>
	<CADk+mPBM3qPHVU91mSh-G175FruVUmT6MaKeHU6iEiz+K2P+DA@mail.gmail.com>
Message-ID: <701C784DE99B0C4C9FDA9EDE66E23E661E4F593C1B@HQ-MB-07.ba.ad.ssa.gov>

Rainer,

I was able to get an extra field in the output file using "annotate" in the rule file, but when I use more than one tag in one rule rsyslog process die (Program terminated with signal 11, Segmentation fault).

I sent detailed information about this issue to rsyslog list (thread:  "log classification with lognorm issue")

Thanks in advance for any help.

Jose

From: lognorm-bounces at lists.adiscon.com [mailto:lognorm-bounces at lists.adiscon.com] On Behalf Of Rainer Gerhards
Sent: Monday, July 15, 2013 3:33 AM
To: lognorm
Subject: Re: [Lognorm] log classification with liblognorm

David made me aware of this thread. Sorry for not responding further, I've been very busy and probably overlooked it. More inline...

On Wed, Jun 26, 2013 at 10:41 PM, Castillo, Jose Contractor <Jose.Castillo at ssa.gov<mailto:Jose.Castillo at ssa.gov>> wrote:
David,

I modified the template:
template(name="testFormat" type="string" string="%$!all-json%,tag='%$!mytag%'")

I also modified the rule to include mytag as a tag:
rule=mytag:%date:word% %host:ipv4% : %%ASA-%number0:number%-%number1:number%: Denied ICMP type=%number2:number%, code=%number3:number% from %origin:ipv4% on interface outside

And now I getting :
{ "origin": "77.2.2.2", "number3": "0", "number2": "8", "number1": "313001", "number0": "3", "host": "0.0.0.0", "date": "2013-06-26T10:47:42+01:00" },tag=''

No tags are being written to the output file.

What am I doing wrong?

I need to check the code, but I think you are doing everything right. IIRC, tags were meant as classification method for rsyslog's engine (if-stmts). I think outputting them was not at the table when this was written. I am 90% sure, and will try to verify the rest within this week.
Rainer

Thanks,

Jose

-----Original Message-----
From: lognorm-bounces at lists.adiscon.com<mailto:lognorm-bounces at lists.adiscon.com> [mailto:lognorm-bounces at lists.adiscon.com] On Behalf Of David Lang
Sent: Wednesday, June 26, 2013 3:11 PM
To: lognorm
Subject: Re: [Lognorm] log classification with liblognorm

the tags are accessed in rsyslog by $!<tag name>

so where you would have done %hostname% you could do %$!mytag%

David Lang

On Wed, 26 Jun 2013, Castillo, Jose   Contractor wrote:

> Date: Wed, 26 Jun 2013 16:00:06 -0400
> From: "Castillo, Jose   Contractor" <Jose.Castillo at ssa.gov<mailto:Jose.Castillo at ssa.gov>>
> Reply-To: lognorm <lognorm at lists.adiscon.com<mailto:lognorm at lists.adiscon.com>>
> To: "lognorm at lists.adiscon.com<mailto:lognorm at lists.adiscon.com>" <lognorm at lists.adiscon.com<mailto:lognorm at lists.adiscon.com>>
> Subject: [Lognorm] log classification with liblognorm
>
> Hello all,
>
> I started to play using rsyslog/liblognorm a week ago, I'm testing them on a CentOS 6.4 virtual machine, and next packages have been installed from Adiscon repository:
> rsyslog-mysql-7.4.1-1.el6.x86_64
> rsyslog-mmjsonparse-7.4.1-1.el6.x86_64
> rsyslog-7.4.1-1.el6.x86_64
> rsyslog-mmnormalize-7.4.1-1.el6.x86_64
> rsyslog-udpspoof-7.4.1-1.el6.x86_64
> rsyslog-elasticsearch-7.4.1-1.el6.x86_64
>
> I was reading next link, and now I'm trying to select subsets of messages based on liblognorm tags.
> http://blog.gerhards.net/2011/04/log-classification-with-liblognorm.ht
> ml
>
> So far, I was able to parse a message using mmnormalize module, and now I want to create outputs based on the liblognorm tag(s).
> Is this option supported within rsyslog.conf? Any example how to do it?
>
> Below my testing files:
>
> =================================================================
> rsyslog.conf file:
> module (load="imudp")
> module (load="mmnormalize")
> module (load="mmjsonparse")
>
> input(type="imudp" address="192.168.1.1" port="514" ruleset="test")
>
> template(name="testFormat" type="string" string="%$!all-json%\n")
>
> ruleset(name="test") {
>        action(type="mmnormalize" userawmsg="on" rulebase="/data/syslog/rulebase.rb")
>        action(type="omfile" file="/data/syslog/test-syslog.log"
> template="testFormat") }
>
> rulebase.rb file:
> rule=test:%date:word% %host:ipv4% :
> %%ASA-%number0:number%-%number1:number%: Denied ICMP
> type=%number2:number%, code=%number3:number% from %origin:ipv4% on
> interface outside
>
> test-syslog.log file:
> { "origin": "77.2.2.2", "number3": "0", "number2": "8", "number1":
> "313001", "number0": "3", "host": "0.0.0.0", "date":
> "2013-06-26T10:47:42+01:00" }
>
> Message:
> "2013-06-26T10:47:42+01:00 0.0.0.0 : %ASA-3-313001: Denied ICMP type=8, code=0 from 77.2.2.2 on interface outside"
> ======================================================================
>
>
> Thanks in advance for any help,
>
> Jose Castillo
>
>
>
>


_______________________________________________
Lognorm mailing list
Lognorm at lists.adiscon.com<mailto:Lognorm at lists.adiscon.com>
http://lists.adiscon.net/mailman/listinfo/lognorm

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20130715/31eaff77/attachment-0001.htm>

From cclark at quadrantsec.com  Mon Jul 15 17:07:36 2013
From: cclark at quadrantsec.com (Champ Clark III)
Date: Mon, 15 Jul 2013 11:07:36 -0400
Subject: [Lognorm] Memory Leak in liblognorm?
In-Reply-To: <51E3F3BB.7050606@quadrantsec.com>
References: <51E06051.5040901@quadrantsec.com>
	<CADk+mPCMKNr+Be7LedbvvdCYnY=L432TMLcTav+wcJpUb+nfKA@mail.gmail.com>
	<51E06187.2010908@quadrantsec.com>
	<CADk+mPBULO7gdETi-Wo3RgPFXBzwyvTNBE_E_yOVdAbtj_qF3A@mail.gmail.com>
	<51E066BE.2010804@jpserver.net> <51E13C55.5000401@quadrantsec.com>
	<CADk+mPDYAQ_4KvU26W9FTogp4NAW8vbYbWguXrpi7yzxAW3pjw@mail.gmail.com>
	<CADk+mPAeLW9rTeLOjheK47wgWCZHF7AzKbyhL-bdcrC8B_Xa6w@mail.gmail.com>
	<51E18A8C.1070502@quadrantsec.com>
	<CADk+mPCCiqrZ0w2ML4PTTa9wOiz3Dg-X6dCffaVO0xYriW-epA@mail.gmail.com>
	<51E1C572.9080003@quadrantsec.com>
	<0f4179f0-c6fc-49d4-86d1-7c835c1c3496@email.android.com>
	<CADk+mPB3AXgQedc4a2tB4yYrKOOLyaZT6YcON6ASM4MwSyiy+A@mail.gmail.com>
	<51E1E508.8040907@quadrantsec.com>
	<CADk+mPD04nm+pHb3h4nH=qgZVojH+s-Xq=xma+kTeTUzLCAYEw@mail.gmail.com>
	<CADk+mPDshYyt4Q_2-Gds3KakAUEx642cacC__Th9JYixLr2DNw@mail.gmail.com>
	<51E3F3BB.7050606@quadrantsec.com>
Message-ID: <51E41038.5030906@quadrantsec.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


Similar to how I was properly cleaning up when dealing with the
"propName",  I tried similar trick.  However,  it didn't work out.


==16533== 892,776 bytes in 8,963 blocks are definitely lost in loss
record 53 of 55
==16533==    at 0x4C2B6CD: malloc (in
/usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==16533==    by 0x5457CD9: es_newStr (string.c:105)
==16533==    by 0x5457D0E: es_newStrFromCStr (string.c:125)
==16533==    by 0x40C2DA: sagan_normalize_liblognorm
(sagan-liblognorm.c:104)
==16533==    by 0x41466F: Sagan_Blacklist (sagan-blacklist.c:167)
==16533==    by 0x40BD37: Sagan_Processor (sagan-processor.c:123)
==16533==    by 0x595EE99: start_thread (pthread_create.c:308)


This is the line:

str = es_newStrFromCStr(syslog_msg, strlen(syslog_msg ));

Which I clear once near the end:

es_deleteStr(str);

I've cleaned up/sync'ed up the code here:

https://github.com/beave/sagan/blob/master/src/sagan-liblognorm.c

Should I be cleaning it up after every usage (for example,  after "str
= ee_getFieldValueAsStr(field, 0);"

I'm confused.    Thanks again!


- -- 
- - Champ Clark III (cclark at quadrantsec.com)
  Quadrant Information Security (http://quadrantsec.com)
  Key Fingerprint: 2E56 C2EB 1B25 C517 D5BA 2DCF 5E70 B2F8 0381 878A
  GPG Key ID: 0381878A


-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJR5BA4AAoJENnmXt7Lmc3K86AH+wbIoTlstbI4C/R7ahTkeVtx
OG37KOsze7tZNfjbBkL47vP+tWGF6goF9I7Dj+eTuzDlfAnLcSxKkmSA7P+xqbjD
pAYrsiU9+ViJK3A2iZhjesK4DUfCKIojNTGTa2O++nyl3hv1Tq3fXpLCDdxeJDoq
F7dphDxRsQxXpfJu7oiapC4JKHNhN3chFpJtbFwBurwDNruNHqBHBinLXNngzmQR
nzmE7eZAoLJXSw+D89OiFydZRZs48DSV4if1EyZBYe/X6Y8PjFht2v+XCjiUYOIQ
J3T+rKtEyau9dPoua4iGm/wK01z/IJAsUcVszR1zeWo6T8HYtZDc27HQUC2nvh0=
=40DE
-----END PGP SIGNATURE-----

From rgerhards at hq.adiscon.com  Mon Jul 15 17:21:01 2013
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 15 Jul 2013 17:21:01 +0200
Subject: [Lognorm] Memory Leak in liblognorm?
In-Reply-To: <51E41038.5030906@quadrantsec.com>
References: <51E06051.5040901@quadrantsec.com>
	<CADk+mPCMKNr+Be7LedbvvdCYnY=L432TMLcTav+wcJpUb+nfKA@mail.gmail.com>
	<51E06187.2010908@quadrantsec.com>
	<CADk+mPBULO7gdETi-Wo3RgPFXBzwyvTNBE_E_yOVdAbtj_qF3A@mail.gmail.com>
	<51E066BE.2010804@jpserver.net> <51E13C55.5000401@quadrantsec.com>
	<CADk+mPDYAQ_4KvU26W9FTogp4NAW8vbYbWguXrpi7yzxAW3pjw@mail.gmail.com>
	<CADk+mPAeLW9rTeLOjheK47wgWCZHF7AzKbyhL-bdcrC8B_Xa6w@mail.gmail.com>
	<51E18A8C.1070502@quadrantsec.com>
	<CADk+mPCCiqrZ0w2ML4PTTa9wOiz3Dg-X6dCffaVO0xYriW-epA@mail.gmail.com>
	<51E1C572.9080003@quadrantsec.com>
	<0f4179f0-c6fc-49d4-86d1-7c835c1c3496@email.android.com>
	<CADk+mPB3AXgQedc4a2tB4yYrKOOLyaZT6YcON6ASM4MwSyiy+A@mail.gmail.com>
	<51E1E508.8040907@quadrantsec.com>
	<CADk+mPD04nm+pHb3h4nH=qgZVojH+s-Xq=xma+kTeTUzLCAYEw@mail.gmail.com>
	<CADk+mPDshYyt4Q_2-Gds3KakAUEx642cacC__Th9JYixLr2DNw@mail.gmail.com>
	<51E3F3BB.7050606@quadrantsec.com>
	<51E41038.5030906@quadrantsec.com>
Message-ID: <CADk+mPCmYQFbo_x3RG7C+EFF3b+0cVUXeTsk6a4=S6LYqw+JdQ@mail.gmail.com>

Sorry a bit short on time, thus brief :-(

On Mon, Jul 15, 2013 at 5:07 PM, Champ Clark III <cclark at quadrantsec.com>wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
> Similar to how I was properly cleaning up when dealing with the
> "propName",  I tried similar trick.  However,  it didn't work out.
>
>
> ==16533== 892,776 bytes in 8,963 blocks are definitely lost in loss
> record 53 of 55
> ==16533==    at 0x4C2B6CD: malloc (in
> /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==16533==    by 0x5457CD9: es_newStr (string.c:105)
> ==16533==    by 0x5457D0E: es_newStrFromCStr (string.c:125)
> ==16533==    by 0x40C2DA: sagan_normalize_liblognorm
> (sagan-liblognorm.c:104)
> ==16533==    by 0x41466F: Sagan_Blacklist (sagan-blacklist.c:167)
> ==16533==    by 0x40BD37: Sagan_Processor (sagan-processor.c:123)
> ==16533==    by 0x595EE99: start_thread (pthread_create.c:308)
>
>
> This is the line:
>
> str = es_newStrFromCStr(syslog_msg, strlen(syslog_msg ));
>
> Which I clear once near the end:
>
> es_deleteStr(str);
>
> I've cleaned up/sync'ed up the code here:
>
> https://github.com/beave/sagan/blob/master/src/sagan-liblognorm.c
>
> Should I be cleaning it up after every usage (for example,  after "str
> = ee_getFieldValueAsStr(field, 0);"
>
>
well, es_newStr() is like malloc.
so

char *s;
s=malloc();
s=malloc();
free(s);

creates a leak. You must free the str object before the ptr is overwritten.

Does that help?

Rainer

> I'm confused.    Thanks again!
>
>
> - --
> - - Champ Clark III (cclark at quadrantsec.com)
>   Quadrant Information Security (http://quadrantsec.com)
>   Key Fingerprint: 2E56 C2EB 1B25 C517 D5BA 2DCF 5E70 B2F8 0381 878A
>   GPG Key ID: 0381878A
>
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
> Comment: GPGTools - http://gpgtools.org
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQEcBAEBAgAGBQJR5BA4AAoJENnmXt7Lmc3K86AH+wbIoTlstbI4C/R7ahTkeVtx
> OG37KOsze7tZNfjbBkL47vP+tWGF6goF9I7Dj+eTuzDlfAnLcSxKkmSA7P+xqbjD
> pAYrsiU9+ViJK3A2iZhjesK4DUfCKIojNTGTa2O++nyl3hv1Tq3fXpLCDdxeJDoq
> F7dphDxRsQxXpfJu7oiapC4JKHNhN3chFpJtbFwBurwDNruNHqBHBinLXNngzmQR
> nzmE7eZAoLJXSw+D89OiFydZRZs48DSV4if1EyZBYe/X6Y8PjFht2v+XCjiUYOIQ
> J3T+rKtEyau9dPoua4iGm/wK01z/IJAsUcVszR1zeWo6T8HYtZDc27HQUC2nvh0=
> =40DE
> -----END PGP SIGNATURE-----
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20130715/c7f0c815/attachment.htm>

From cclark at quadrantsec.com  Mon Jul 15 18:00:57 2013
From: cclark at quadrantsec.com (Champ Clark III)
Date: Mon, 15 Jul 2013 12:00:57 -0400
Subject: [Lognorm] Memory Leak in liblognorm?
In-Reply-To: <CADk+mPCmYQFbo_x3RG7C+EFF3b+0cVUXeTsk6a4=S6LYqw+JdQ@mail.gmail.com>
References: <51E06051.5040901@quadrantsec.com>
	<51E06187.2010908@quadrantsec.com>
	<CADk+mPBULO7gdETi-Wo3RgPFXBzwyvTNBE_E_yOVdAbtj_qF3A@mail.gmail.com>
	<51E066BE.2010804@jpserver.net> <51E13C55.5000401@quadrantsec.com>
	<CADk+mPDYAQ_4KvU26W9FTogp4NAW8vbYbWguXrpi7yzxAW3pjw@mail.gmail.com>
	<CADk+mPAeLW9rTeLOjheK47wgWCZHF7AzKbyhL-bdcrC8B_Xa6w@mail.gmail.com>
	<51E18A8C.1070502@quadrantsec.com>
	<CADk+mPCCiqrZ0w2ML4PTTa9wOiz3Dg-X6dCffaVO0xYriW-epA@mail.gmail.com>
	<51E1C572.9080003@quadrantsec.com>
	<0f4179f0-c6fc-49d4-86d1-7c835c1c3496@email.android.com>
	<CADk+mPB3AXgQedc4a2tB4yYrKOOLyaZT6YcON6ASM4MwSyiy+A@mail.gmail.com>
	<51E1E508.8040907@quadrantsec.com>
	<CADk+mPD04nm+pHb3h4nH=qgZVojH+s-Xq=xma+kTeTUzLCAYEw@mail.gmail.com>
	<CADk+mPDshYyt4Q_2-Gds3KakAUEx642cacC__Th9JYixLr2DNw@mail.gmail.com>
	<51E3F3BB.7050606@quadrantsec.com>
	<51E41038.5030906@quadrantsec.com>
	<CADk+mPCmYQFbo_x3RG7C+EFF3b+0cVUXeTsk6a4=S6LYqw+JdQ@mail.gmail.com>
Message-ID: <51E41CB9.7030406@quadrantsec.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I know you are a busy guy,  so no rush :)

>>> well, es_newStr() is like malloc.
>> so
> 
>> char *s; s=malloc(); s=malloc(); free(s);
> 
>> creates a leak. You must free the str object before the ptr is
>> overwritten. Does that help?


Well, not really.

I call "es_newStrFromCStr" one time,  and then call
"es_deleteStr(str);" one time (at the end).   So I don't think the
malloc example applies.

I'll keep plugging away at it.


- -- 
- - Champ Clark III (cclark at quadrantsec.com)
  Quadrant Information Security (http://quadrantsec.com)
  Key Fingerprint: 2E56 C2EB 1B25 C517 D5BA 2DCF 5E70 B2F8 0381 878A
  GPG Key ID: 0381878A
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJR5By5AAoJENnmXt7Lmc3KXO0H/0FqRj13KFdFjf18DddGJ6RD
gyH0NxNwMkFmBEhi3lXj7AyVAq2+UxinqXAAYwE53qCqiELCoDwY4D3zjv84aRsD
wZIGsQDmtWLZ60vY/wFZRFGFxZxXySxJ3czNmGPON7R1nKzxXsaKQQu9rej5B+BW
Bh2TFWqcva29UAko+Rfx5hMAOanDUf0l3trWOf5VRCk9VTP+e4cMIHT3DkToXTc/
5AIa2AgMQYR0lOb02qQAKbYCSVvLQDnBWn7fYanNbLNB4+Vysa8MKdVvmXy8Yd1B
YSyUzxPsOz7JYJD3WrDqONPnfiJEbRKW67mpqG0BLsb/k2qzQZ42HK+54mqOmqs=
=l+On
-----END PGP SIGNATURE-----

From cclark at quadrantsec.com  Mon Jul 15 21:06:03 2013
From: cclark at quadrantsec.com (Champ Clark III)
Date: Mon, 15 Jul 2013 15:06:03 -0400
Subject: [Lognorm] Memory Leak in liblognorm?
In-Reply-To: <CADk+mPCmYQFbo_x3RG7C+EFF3b+0cVUXeTsk6a4=S6LYqw+JdQ@mail.gmail.com>
References: <51E06051.5040901@quadrantsec.com>
	<51E06187.2010908@quadrantsec.com>
	<CADk+mPBULO7gdETi-Wo3RgPFXBzwyvTNBE_E_yOVdAbtj_qF3A@mail.gmail.com>
	<51E066BE.2010804@jpserver.net> <51E13C55.5000401@quadrantsec.com>
	<CADk+mPDYAQ_4KvU26W9FTogp4NAW8vbYbWguXrpi7yzxAW3pjw@mail.gmail.com>
	<CADk+mPAeLW9rTeLOjheK47wgWCZHF7AzKbyhL-bdcrC8B_Xa6w@mail.gmail.com>
	<51E18A8C.1070502@quadrantsec.com>
	<CADk+mPCCiqrZ0w2ML4PTTa9wOiz3Dg-X6dCffaVO0xYriW-epA@mail.gmail.com>
	<51E1C572.9080003@quadrantsec.com>
	<0f4179f0-c6fc-49d4-86d1-7c835c1c3496@email.android.com>
	<CADk+mPB3AXgQedc4a2tB4yYrKOOLyaZT6YcON6ASM4MwSyiy+A@mail.gmail.com>
	<51E1E508.8040907@quadrantsec.com>
	<CADk+mPD04nm+pHb3h4nH=qgZVojH+s-Xq=xma+kTeTUzLCAYEw@mail.gmail.com>
	<CADk+mPDshYyt4Q_2-Gds3KakAUEx642cacC__Th9JYixLr2DNw@mail.gmail.com>
	<51E3F3BB.7050606@quadrantsec.com>
	<51E41038.5030906@quadrantsec.com>
	<CADk+mPCmYQFbo_x3RG7C+EFF3b+0cVUXeTsk6a4=S6LYqw+JdQ@mail.gmail.com>
Message-ID: <51E4481B.7000304@quadrantsec.com>


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


Hello,

So - I've stripped down the code a good bit to see if I can't isolate
where I'm going wrong.  Below is what I got:

- --<snip>--
str = es_newStrFromCStr(syslog_msg, strlen(syslog_msg));
ln_normalize(ctx, str, &lnevent);

       if(lnevent != NULL) {
                es_emptyStr(str);
                ee_fmtEventToRFC5424(lnevent, &str);
                }

free(cstr);
es_deleteStr(str);
ee_deleteEvent(lnevent);
}
- --<snip>--

It appears as soon as I add the "ee_fmtEventToRFC5424",  valgrind starts
to report the following:

==21979== 69,872 bytes in 614 blocks are definitely lost in loss record
52 of 54
==21979==    at 0x4C2B6CD: malloc (in
/usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==21979==    by 0x5457CD9: es_newStr (string.c:105)
==21979==    by 0x5457D0E: es_newStrFromCStr (string.c:125)
==21979==    by 0x40C167: sagan_normalize_liblognorm
(sagan-liblognorm.c:103)
==21979==    by 0x41427F: Sagan_Blacklist (sagan-blacklist.c:167)
==21979==    by 0x40BC07: Sagan_Processor (sagan-processor.c:123)
==21979==    by 0x595EE99: start_thread (pthread_create.c:308)

If I remove the line,  that goes away.    Any thoughts?

Thanks for your time.

- -- 
- - Quadrant Information Security
  Champ Clark III
  o: 800.538.9357 x 101
  c: 850.443.2440
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJR5EgaAAoJENnmXt7Lmc3K6qMH/2ohomzUEQr+yi6pwNRUbM6P
P3LfzxqccDSsJkDyY5EYY611S8ohHP0ZRdFyzQmRger1KXfSgvEGbs+brZC8rKYb
OINB89GzG7A/Wu3SHnDwNSeLC2OKT+e14FslgBH0PqPgLy7PWnUmXbLxVAs58DXH
3XydPkLumZC2K/vQwYFfxMxGTM90+cE8QRUXdaQ5ihndUg/9zI5BPeOvrOOffR69
ZscCxZhob/qWMlCuOZd37iAayMROnzg8dWFq1SoWiITFn+PazhyXVsXvbA823VYJ
vztu09FeH3LkcKt0XNgY8a+ITBmSTZ65ioMnJ4sXRjk2Sh1M94fWl8NV2sDX1qk=
=oOrC
-----END PGP SIGNATURE-----


From rgerhards at hq.adiscon.com  Mon Jul 15 21:46:39 2013
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 15 Jul 2013 21:46:39 +0200
Subject: [Lognorm] Memory Leak in liblognorm?
In-Reply-To: <51E4481B.7000304@quadrantsec.com>
References: <51E06051.5040901@quadrantsec.com>
	<51E06187.2010908@quadrantsec.com>
	<CADk+mPBULO7gdETi-Wo3RgPFXBzwyvTNBE_E_yOVdAbtj_qF3A@mail.gmail.com>
	<51E066BE.2010804@jpserver.net> <51E13C55.5000401@quadrantsec.com>
	<CADk+mPDYAQ_4KvU26W9FTogp4NAW8vbYbWguXrpi7yzxAW3pjw@mail.gmail.com>
	<CADk+mPAeLW9rTeLOjheK47wgWCZHF7AzKbyhL-bdcrC8B_Xa6w@mail.gmail.com>
	<51E18A8C.1070502@quadrantsec.com>
	<CADk+mPCCiqrZ0w2ML4PTTa9wOiz3Dg-X6dCffaVO0xYriW-epA@mail.gmail.com>
	<51E1C572.9080003@quadrantsec.com>
	<0f4179f0-c6fc-49d4-86d1-7c835c1c3496@email.android.com>
	<CADk+mPB3AXgQedc4a2tB4yYrKOOLyaZT6YcON6ASM4MwSyiy+A@mail.gmail.com>
	<51E1E508.8040907@quadrantsec.com>
	<CADk+mPD04nm+pHb3h4nH=qgZVojH+s-Xq=xma+kTeTUzLCAYEw@mail.gmail.com>
	<CADk+mPDshYyt4Q_2-Gds3KakAUEx642cacC__Th9JYixLr2DNw@mail.gmail.com>
	<51E3F3BB.7050606@quadrantsec.com>
	<51E41038.5030906@quadrantsec.com>
	<CADk+mPCmYQFbo_x3RG7C+EFF3b+0cVUXeTsk6a4=S6LYqw+JdQ@mail.gmail.com>
	<51E4481B.7000304@quadrantsec.com>
Message-ID: <CADk+mPAsRV-uPBwtriFX13EoDpsdHkWUMT-Fuaw0YSieZdv-cA@mail.gmail.com>

Use es_deleteStr instead of es_emptyStr. The latter just resets it but does
not free.  More explanations follow tomorrow.  Please report back.

Sent from phone, thus brief.
Am 15.07.2013 21:06 schrieb "Champ Clark III" <cclark at quadrantsec.com>:

>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
> Hello,
>
> So - I've stripped down the code a good bit to see if I can't isolate
> where I'm going wrong.  Below is what I got:
>
> - --<snip>--
> str = es_newStrFromCStr(syslog_msg, strlen(syslog_msg));
> ln_normalize(ctx, str, &lnevent);
>
>        if(lnevent != NULL) {
>                 es_emptyStr(str);
>                 ee_fmtEventToRFC5424(lnevent, &str);
>                 }
>
> free(cstr);
> es_deleteStr(str);
> ee_deleteEvent(lnevent);
> }
> - --<snip>--
>
> It appears as soon as I add the "ee_fmtEventToRFC5424",  valgrind starts
> to report the following:
>
> ==21979== 69,872 bytes in 614 blocks are definitely lost in loss record
> 52 of 54
> ==21979==    at 0x4C2B6CD: malloc (in
> /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==21979==    by 0x5457CD9: es_newStr (string.c:105)
> ==21979==    by 0x5457D0E: es_newStrFromCStr (string.c:125)
> ==21979==    by 0x40C167: sagan_normalize_liblognorm
> (sagan-liblognorm.c:103)
> ==21979==    by 0x41427F: Sagan_Blacklist (sagan-blacklist.c:167)
> ==21979==    by 0x40BC07: Sagan_Processor (sagan-processor.c:123)
> ==21979==    by 0x595EE99: start_thread (pthread_create.c:308)
>
> If I remove the line,  that goes away.    Any thoughts?
>
> Thanks for your time.
>
> - --
> - - Quadrant Information Security
>   Champ Clark III
>   o: 800.538.9357 x 101
>   c: 850.443.2440
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (GNU/Linux)
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQEcBAEBAgAGBQJR5EgaAAoJENnmXt7Lmc3K6qMH/2ohomzUEQr+yi6pwNRUbM6P
> P3LfzxqccDSsJkDyY5EYY611S8ohHP0ZRdFyzQmRger1KXfSgvEGbs+brZC8rKYb
> OINB89GzG7A/Wu3SHnDwNSeLC2OKT+e14FslgBH0PqPgLy7PWnUmXbLxVAs58DXH
> 3XydPkLumZC2K/vQwYFfxMxGTM90+cE8QRUXdaQ5ihndUg/9zI5BPeOvrOOffR69
> ZscCxZhob/qWMlCuOZd37iAayMROnzg8dWFq1SoWiITFn+PazhyXVsXvbA823VYJ
> vztu09FeH3LkcKt0XNgY8a+ITBmSTZ65ioMnJ4sXRjk2Sh1M94fWl8NV2sDX1qk=
> =oOrC
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20130715/a8c31d49/attachment-0001.htm>

From cclark at quadrantsec.com  Mon Jul 15 22:02:53 2013
From: cclark at quadrantsec.com (Champ Clark III)
Date: Mon, 15 Jul 2013 16:02:53 -0400
Subject: [Lognorm] Memory Leak in liblognorm?
In-Reply-To: <CADk+mPAsRV-uPBwtriFX13EoDpsdHkWUMT-Fuaw0YSieZdv-cA@mail.gmail.com>
References: <51E06051.5040901@quadrantsec.com> <51E066BE.2010804@jpserver.net>
	<51E13C55.5000401@quadrantsec.com>
	<CADk+mPDYAQ_4KvU26W9FTogp4NAW8vbYbWguXrpi7yzxAW3pjw@mail.gmail.com>
	<CADk+mPAeLW9rTeLOjheK47wgWCZHF7AzKbyhL-bdcrC8B_Xa6w@mail.gmail.com>
	<51E18A8C.1070502@quadrantsec.com>
	<CADk+mPCCiqrZ0w2ML4PTTa9wOiz3Dg-X6dCffaVO0xYriW-epA@mail.gmail.com>
	<51E1C572.9080003@quadrantsec.com>
	<0f4179f0-c6fc-49d4-86d1-7c835c1c3496@email.android.com>
	<CADk+mPB3AXgQedc4a2tB4yYrKOOLyaZT6YcON6ASM4MwSyiy+A@mail.gmail.com>
	<51E1E508.8040907@quadrantsec.com>
	<CADk+mPD04nm+pHb3h4nH=qgZVojH+s-Xq=xma+kTeTUzLCAYEw@mail.gmail.com>
	<CADk+mPDshYyt4Q_2-Gds3KakAUEx642cacC__Th9JYixLr2DNw@mail.gmail.com>
	<51E3F3BB.7050606@quadrantsec.com>
	<51E41038.5030906@quadrantsec.com>
	<CADk+mPCmYQFbo_x3RG7C+EFF3b+0cVUXeTsk6a4=S6LYqw+JdQ@mail.gmail.com>
	<51E4481B.7000304@quadrantsec.com>
	<CADk+mPAsRV-uPBwtriFX13EoDpsdHkWUMT-Fuaw0YSieZdv-cA@mail.gmail.com>
Message-ID: <51E4556D.3030706@quadrantsec.com>


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Sorry,  but same results :(   I'm using the same test code below but
with es_emptyStr(str) replaced with es_deleteStr(str)



On 07/15/2013 03:46 PM, Rainer Gerhards wrote:
>
> Use es_deleteStr instead of es_emptyStr. The latter just resets it but
does not free.  More explanations follow tomorrow.  Please report back.
>
> Sent from phone, thus brief.
>
> Am 15.07.2013 21:06 schrieb "Champ Clark III" <cclark at quadrantsec.com
<mailto:cclark at quadrantsec.com>>:
>
>
>
> Hello,
>
> So - I've stripped down the code a good bit to see if I can't isolate
> where I'm going wrong.  Below is what I got:
>
> --<snip>--
> str = es_newStrFromCStr(syslog_msg, strlen(syslog_msg));
> ln_normalize(ctx, str, &lnevent);
>
>        if(lnevent != NULL) {
>                 es_emptyStr(str);
>                 ee_fmtEventToRFC5424(lnevent, &str);
>                 }
>
> free(cstr);
> es_deleteStr(str);
> ee_deleteEvent(lnevent);
> }
> --<snip>--
>
> It appears as soon as I add the "ee_fmtEventToRFC5424",  valgrind starts
> to report the following:
>
> ==21979== 69,872 bytes in 614 blocks are definitely lost in loss record
> 52 of 54
> ==21979==    at 0x4C2B6CD: malloc (in
> /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==21979==    by 0x5457CD9: es_newStr (string.c:105)
> ==21979==    by 0x5457D0E: es_newStrFromCStr (string.c:125)
> ==21979==    by 0x40C167: sagan_normalize_liblognorm
> (sagan-liblognorm.c:103)
> ==21979==    by 0x41427F: Sagan_Blacklist (sagan-blacklist.c:167)
> ==21979==    by 0x40BC07: Sagan_Processor (sagan-processor.c:123)
> ==21979==    by 0x595EE99: start_thread (pthread_create.c:308)
>
> If I remove the line,  that goes away.    Any thoughts?
>
> Thanks for your time.
>
>
>     _______________________________________________
>     Lognorm mailing list
>     Lognorm at lists.adiscon.com <mailto:Lognorm at lists.adiscon.com>
>     http://lists.adiscon.net/mailman/listinfo/lognorm
>
>
>
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm

- -- 
- - Quadrant Information Security
  Champ Clark III
  o: 800.538.9357 x 101
  c: 850.443.2440
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJR5FVtAAoJENnmXt7Lmc3K6kQH/j1eiHNggCJMRys8vkiEeKme
evsfBZpTzlw7iTJFsHv/d3vBJub9dcm80y8kgYvmYOrjSea+GIWYKP8a1Ij44b/x
fwd6QQYzRQ0VZljCFTbW+1xAczzaqTiaJyd4lljG3ecu7BFGG61FwWkr91DU+S1L
DjhqAONjitYvJC4ZdHNoDfBuodBLTT0qr/BHtbYFDFCYfsIK8+7aSPw3PO6sFM/5
rjGDXOcXYXZ8niPUorR0CdNekaqzUC5nwj3hUJBYWlw23FTzQfBuI+rIj2IfNl8Z
6LIJXtfpbGt9z0eHOwnVRbytGKgOWheX2o27DGHSlcy41myIixgTZ6We2yNGk/s=
=Ukse
-----END PGP SIGNATURE-----

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20130715/66af66e0/attachment.htm>

From rgerhards at hq.adiscon.com  Mon Jul 15 22:05:21 2013
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 15 Jul 2013 22:05:21 +0200
Subject: [Lognorm] Memory Leak in liblognorm?
In-Reply-To: <51E4556D.3030706@quadrantsec.com>
References: <51E06051.5040901@quadrantsec.com> <51E066BE.2010804@jpserver.net>
	<51E13C55.5000401@quadrantsec.com>
	<CADk+mPDYAQ_4KvU26W9FTogp4NAW8vbYbWguXrpi7yzxAW3pjw@mail.gmail.com>
	<CADk+mPAeLW9rTeLOjheK47wgWCZHF7AzKbyhL-bdcrC8B_Xa6w@mail.gmail.com>
	<51E18A8C.1070502@quadrantsec.com>
	<CADk+mPCCiqrZ0w2ML4PTTa9wOiz3Dg-X6dCffaVO0xYriW-epA@mail.gmail.com>
	<51E1C572.9080003@quadrantsec.com>
	<0f4179f0-c6fc-49d4-86d1-7c835c1c3496@email.android.com>
	<CADk+mPB3AXgQedc4a2tB4yYrKOOLyaZT6YcON6ASM4MwSyiy+A@mail.gmail.com>
	<51E1E508.8040907@quadrantsec.com>
	<CADk+mPD04nm+pHb3h4nH=qgZVojH+s-Xq=xma+kTeTUzLCAYEw@mail.gmail.com>
	<CADk+mPDshYyt4Q_2-Gds3KakAUEx642cacC__Th9JYixLr2DNw@mail.gmail.com>
	<51E3F3BB.7050606@quadrantsec.com>
	<51E41038.5030906@quadrantsec.com>
	<CADk+mPCmYQFbo_x3RG7C+EFF3b+0cVUXeTsk6a4=S6LYqw+JdQ@mail.gmail.com>
	<51E4481B.7000304@quadrantsec.com>
	<CADk+mPAsRV-uPBwtriFX13EoDpsdHkWUMT-Fuaw0YSieZdv-cA@mail.gmail.com>
	<51E4556D.3030706@quadrantsec.com>
Message-ID: <CADk+mPCweXVJ-WFDhXtzGm_x5vsCFB1SYptn0i1K8a=tBcC_mQ@mail.gmail.com>

Ok thx.  Than i need to check the API  spec.  I have seen that the web doc
is pretty old and missing functions - hopefully only the web...

Sent from phone, thus brief.
Am 15.07.2013 22:02 schrieb "Champ Clark III" <cclark at quadrantsec.com>:

>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Sorry,  but same results :(   I'm using the same test code below but with
> es_emptyStr(str) replaced with es_deleteStr(str)
>
>
>
> On 07/15/2013 03:46 PM, Rainer Gerhards wrote:
> >
> > Use es_deleteStr instead of es_emptyStr. The latter just resets it but
> does not free.  More explanations follow tomorrow.  Please report back.
> >
> > Sent from phone, thus brief.
> >
> > Am 15.07.2013 21:06 schrieb "Champ Clark III" <cclark at quadrantsec.com
> <mailto:cclark at quadrantsec.com> <cclark at quadrantsec.com>>:
> >
> >
> >
> > Hello,
> >
> > So - I've stripped down the code a good bit to see if I can't isolate
> > where I'm going wrong.  Below is what I got:
> >
> > --<snip>--
> > str = es_newStrFromCStr(syslog_msg, strlen(syslog_msg));
> > ln_normalize(ctx, str, &lnevent);
> >
> >        if(lnevent != NULL) {
> >                 es_emptyStr(str);
> >                 ee_fmtEventToRFC5424(lnevent, &str);
> >                 }
> >
> > free(cstr);
> > es_deleteStr(str);
> > ee_deleteEvent(lnevent);
> > }
> > --<snip>--
> >
> > It appears as soon as I add the "ee_fmtEventToRFC5424",  valgrind starts
> > to report the following:
> >
> > ==21979== 69,872 bytes in 614 blocks are definitely lost in loss record
> > 52 of 54
> > ==21979==    at 0x4C2B6CD: malloc (in
> > /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
> > ==21979==    by 0x5457CD9: es_newStr (string.c:105)
> > ==21979==    by 0x5457D0E: es_newStrFromCStr (string.c:125)
> > ==21979==    by 0x40C167: sagan_normalize_liblognorm
> > (sagan-liblognorm.c:103)
> > ==21979==    by 0x41427F: Sagan_Blacklist (sagan-blacklist.c:167)
> > ==21979==    by 0x40BC07: Sagan_Processor (sagan-processor.c:123)
> > ==21979==    by 0x595EE99: start_thread (pthread_create.c:308)
> >
> > If I remove the line,  that goes away.    Any thoughts?
> >
> > Thanks for your time.
> >
> >
> >     _______________________________________________
> >     Lognorm mailing list
> >     Lognorm at lists.adiscon.com <mailto:Lognorm at lists.adiscon.com><Lognorm at lists.adiscon.com>
> >     http://lists.adiscon.net/mailman/listinfo/lognorm
> >
> >
> >
> > _______________________________________________
> > Lognorm mailing list
> > Lognorm at lists.adiscon.com
> > http://lists.adiscon.net/mailman/listinfo/lognorm
>
> - --
> - - Quadrant Information Security
>   Champ Clark III
>   o: 800.538.9357 x 101
>   c: 850.443.2440
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (GNU/Linux)
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQEcBAEBAgAGBQJR5FVtAAoJENnmXt7Lmc3K6kQH/j1eiHNggCJMRys8vkiEeKme
> evsfBZpTzlw7iTJFsHv/d3vBJub9dcm80y8kgYvmYOrjSea+GIWYKP8a1Ij44b/x
> fwd6QQYzRQ0VZljCFTbW+1xAczzaqTiaJyd4lljG3ecu7BFGG61FwWkr91DU+S1L
> DjhqAONjitYvJC4ZdHNoDfBuodBLTT0qr/BHtbYFDFCYfsIK8+7aSPw3PO6sFM/5
> rjGDXOcXYXZ8niPUorR0CdNekaqzUC5nwj3hUJBYWlw23FTzQfBuI+rIj2IfNl8Z
> 6LIJXtfpbGt9z0eHOwnVRbytGKgOWheX2o27DGHSlcy41myIixgTZ6We2yNGk/s=
> =Ukse
> -----END PGP SIGNATURE-----
>
>
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20130715/40b5586d/attachment.htm>

From rgerhards at hq.adiscon.com  Mon Jul 15 22:06:20 2013
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 15 Jul 2013 22:06:20 +0200
Subject: [Lognorm] Memory Leak in liblognorm?
In-Reply-To: <51E4556D.3030706@quadrantsec.com>
References: <51E06051.5040901@quadrantsec.com> <51E066BE.2010804@jpserver.net>
	<51E13C55.5000401@quadrantsec.com>
	<CADk+mPDYAQ_4KvU26W9FTogp4NAW8vbYbWguXrpi7yzxAW3pjw@mail.gmail.com>
	<CADk+mPAeLW9rTeLOjheK47wgWCZHF7AzKbyhL-bdcrC8B_Xa6w@mail.gmail.com>
	<51E18A8C.1070502@quadrantsec.com>
	<CADk+mPCCiqrZ0w2ML4PTTa9wOiz3Dg-X6dCffaVO0xYriW-epA@mail.gmail.com>
	<51E1C572.9080003@quadrantsec.com>
	<0f4179f0-c6fc-49d4-86d1-7c835c1c3496@email.android.com>
	<CADk+mPB3AXgQedc4a2tB4yYrKOOLyaZT6YcON6ASM4MwSyiy+A@mail.gmail.com>
	<51E1E508.8040907@quadrantsec.com>
	<CADk+mPD04nm+pHb3h4nH=qgZVojH+s-Xq=xma+kTeTUzLCAYEw@mail.gmail.com>
	<CADk+mPDshYyt4Q_2-Gds3KakAUEx642cacC__Th9JYixLr2DNw@mail.gmail.com>
	<51E3F3BB.7050606@quadrantsec.com>
	<51E41038.5030906@quadrantsec.com>
	<CADk+mPCmYQFbo_x3RG7C+EFF3b+0cVUXeTsk6a4=S6LYqw+JdQ@mail.gmail.com>
	<51E4481B.7000304@quadrantsec.com>
	<CADk+mPAsRV-uPBwtriFX13EoDpsdHkWUMT-Fuaw0YSieZdv-cA@mail.gmail.com>
	<51E4556D.3030706@quadrantsec.com>
Message-ID: <CADk+mPBDcM-PQqpYsnczVuAgKYME0iTXMyctSHQrfzuqfKG0nA@mail.gmail.com>

I forgot: is your test code available online?

Sent from phone, thus brief.
Am 15.07.2013 22:02 schrieb "Champ Clark III" <cclark at quadrantsec.com>:

>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Sorry,  but same results :(   I'm using the same test code below but with
> es_emptyStr(str) replaced with es_deleteStr(str)
>
>
>
> On 07/15/2013 03:46 PM, Rainer Gerhards wrote:
> >
> > Use es_deleteStr instead of es_emptyStr. The latter just resets it but
> does not free.  More explanations follow tomorrow.  Please report back.
> >
> > Sent from phone, thus brief.
> >
> > Am 15.07.2013 21:06 schrieb "Champ Clark III" <cclark at quadrantsec.com
> <mailto:cclark at quadrantsec.com> <cclark at quadrantsec.com>>:
> >
> >
> >
> > Hello,
> >
> > So - I've stripped down the code a good bit to see if I can't isolate
> > where I'm going wrong.  Below is what I got:
> >
> > --<snip>--
> > str = es_newStrFromCStr(syslog_msg, strlen(syslog_msg));
> > ln_normalize(ctx, str, &lnevent);
> >
> >        if(lnevent != NULL) {
> >                 es_emptyStr(str);
> >                 ee_fmtEventToRFC5424(lnevent, &str);
> >                 }
> >
> > free(cstr);
> > es_deleteStr(str);
> > ee_deleteEvent(lnevent);
> > }
> > --<snip>--
> >
> > It appears as soon as I add the "ee_fmtEventToRFC5424",  valgrind starts
> > to report the following:
> >
> > ==21979== 69,872 bytes in 614 blocks are definitely lost in loss record
> > 52 of 54
> > ==21979==    at 0x4C2B6CD: malloc (in
> > /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
> > ==21979==    by 0x5457CD9: es_newStr (string.c:105)
> > ==21979==    by 0x5457D0E: es_newStrFromCStr (string.c:125)
> > ==21979==    by 0x40C167: sagan_normalize_liblognorm
> > (sagan-liblognorm.c:103)
> > ==21979==    by 0x41427F: Sagan_Blacklist (sagan-blacklist.c:167)
> > ==21979==    by 0x40BC07: Sagan_Processor (sagan-processor.c:123)
> > ==21979==    by 0x595EE99: start_thread (pthread_create.c:308)
> >
> > If I remove the line,  that goes away.    Any thoughts?
> >
> > Thanks for your time.
> >
> >
> >     _______________________________________________
> >     Lognorm mailing list
> >     Lognorm at lists.adiscon.com <mailto:Lognorm at lists.adiscon.com><Lognorm at lists.adiscon.com>
> >     http://lists.adiscon.net/mailman/listinfo/lognorm
> >
> >
> >
> > _______________________________________________
> > Lognorm mailing list
> > Lognorm at lists.adiscon.com
> > http://lists.adiscon.net/mailman/listinfo/lognorm
>
> - --
> - - Quadrant Information Security
>   Champ Clark III
>   o: 800.538.9357 x 101
>   c: 850.443.2440
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (GNU/Linux)
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQEcBAEBAgAGBQJR5FVtAAoJENnmXt7Lmc3K6kQH/j1eiHNggCJMRys8vkiEeKme
> evsfBZpTzlw7iTJFsHv/d3vBJub9dcm80y8kgYvmYOrjSea+GIWYKP8a1Ij44b/x
> fwd6QQYzRQ0VZljCFTbW+1xAczzaqTiaJyd4lljG3ecu7BFGG61FwWkr91DU+S1L
> DjhqAONjitYvJC4ZdHNoDfBuodBLTT0qr/BHtbYFDFCYfsIK8+7aSPw3PO6sFM/5
> rjGDXOcXYXZ8niPUorR0CdNekaqzUC5nwj3hUJBYWlw23FTzQfBuI+rIj2IfNl8Z
> 6LIJXtfpbGt9z0eHOwnVRbytGKgOWheX2o27DGHSlcy41myIixgTZ6We2yNGk/s=
> =Ukse
> -----END PGP SIGNATURE-----
>
>
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20130715/0a99441c/attachment-0001.htm>

From cclark at quadrantsec.com  Mon Jul 15 22:08:39 2013
From: cclark at quadrantsec.com (Champ Clark III)
Date: Mon, 15 Jul 2013 16:08:39 -0400
Subject: [Lognorm] Memory Leak in liblognorm?
In-Reply-To: <CADk+mPCweXVJ-WFDhXtzGm_x5vsCFB1SYptn0i1K8a=tBcC_mQ@mail.gmail.com>
References: <51E06051.5040901@quadrantsec.com>
	<CADk+mPDYAQ_4KvU26W9FTogp4NAW8vbYbWguXrpi7yzxAW3pjw@mail.gmail.com>
	<CADk+mPAeLW9rTeLOjheK47wgWCZHF7AzKbyhL-bdcrC8B_Xa6w@mail.gmail.com>
	<51E18A8C.1070502@quadrantsec.com>
	<CADk+mPCCiqrZ0w2ML4PTTa9wOiz3Dg-X6dCffaVO0xYriW-epA@mail.gmail.com>
	<51E1C572.9080003@quadrantsec.com>
	<0f4179f0-c6fc-49d4-86d1-7c835c1c3496@email.android.com>
	<CADk+mPB3AXgQedc4a2tB4yYrKOOLyaZT6YcON6ASM4MwSyiy+A@mail.gmail.com>
	<51E1E508.8040907@quadrantsec.com>
	<CADk+mPD04nm+pHb3h4nH=qgZVojH+s-Xq=xma+kTeTUzLCAYEw@mail.gmail.com>
	<CADk+mPDshYyt4Q_2-Gds3KakAUEx642cacC__Th9JYixLr2DNw@mail.gmail.com>
	<51E3F3BB.7050606@quadrantsec.com>
	<51E41038.5030906@quadrantsec.com>
	<CADk+mPCmYQFbo_x3RG7C+EFF3b+0cVUXeTsk6a4=S6LYqw+JdQ@mail.gmail.com>
	<51E4481B.7000304@quadrantsec.com>
	<CADk+mPAsRV-uPBwtriFX13EoDpsdHkWUMT-Fuaw0YSieZdv-cA@mail.gmail.com>
	<51E4556D.3030706@quadrantsec.com>
	<CADk+mPCweXVJ-WFDhXtzGm_x5vsCFB1SYptn0i1K8a=tBcC_mQ@mail.gmail.com>
Message-ID: <51E456C7.2@quadrantsec.com>


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I moved the code over to a pretty busy box which allows me to replicate
pretty quickly.  If you need anything
from me,  just let me know.    I'll continue poking around with it :)


On 07/15/2013 04:05 PM, Rainer Gerhards wrote:
>
> Ok thx.  Than i need to check the API  spec.  I have seen that the web
doc is pretty old and missing functions - hopefully only the web...
>
> Sent from phone, thus brief.
>
> Am 15.07.2013 22:02 schrieb "Champ Clark III" <cclark at quadrantsec.com
<mailto:cclark at quadrantsec.com>>:
>

- -- 
- - Quadrant Information Security
  Champ Clark III
  o: 800.538.9357 x 101
  c: 850.443.2440
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJR5FbHAAoJENnmXt7Lmc3K7vUH/0eq2hYbfPMdmh+Kr/kXGkAB
h5rB8FO++Q46id9UUhDJfcFuPINGSnzzfFV5Hw1g/HD+TvRXuwgslyrbodFfjxM3
fJhKLcuJ834oDugwAvJRORiwvVFMZa9UA03j5zOEnbT7j0m9pbDpqThh/ERBGQNn
DlqAZ4IjSqwU1E/k2Hhc5DDHTg9xEQYrPRd+nyZoYBEo0YXzjNCkOEWke3MnP6zn
nO8pHMd2Ua9495iMpbmpiavIkwVh98VhPblOw1Czgd7OyWSQLjSq2R98uspSqE6T
nk0rHOaKxEI1yjV8Sn59Ay7tXteQlpcPLClxFEHde844lNvetVTtHPykmTzsqx8=
=aeHg
-----END PGP SIGNATURE-----

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20130715/7d5b2d50/attachment.htm>

From cclark at quadrantsec.com  Mon Jul 15 22:10:10 2013
From: cclark at quadrantsec.com (Champ Clark III)
Date: Mon, 15 Jul 2013 16:10:10 -0400
Subject: [Lognorm] Memory Leak in liblognorm?
In-Reply-To: <CADk+mPBDcM-PQqpYsnczVuAgKYME0iTXMyctSHQrfzuqfKG0nA@mail.gmail.com>
References: <51E06051.5040901@quadrantsec.com>
	<CADk+mPDYAQ_4KvU26W9FTogp4NAW8vbYbWguXrpi7yzxAW3pjw@mail.gmail.com>
	<CADk+mPAeLW9rTeLOjheK47wgWCZHF7AzKbyhL-bdcrC8B_Xa6w@mail.gmail.com>
	<51E18A8C.1070502@quadrantsec.com>
	<CADk+mPCCiqrZ0w2ML4PTTa9wOiz3Dg-X6dCffaVO0xYriW-epA@mail.gmail.com>
	<51E1C572.9080003@quadrantsec.com>
	<0f4179f0-c6fc-49d4-86d1-7c835c1c3496@email.android.com>
	<CADk+mPB3AXgQedc4a2tB4yYrKOOLyaZT6YcON6ASM4MwSyiy+A@mail.gmail.com>
	<51E1E508.8040907@quadrantsec.com>
	<CADk+mPD04nm+pHb3h4nH=qgZVojH+s-Xq=xma+kTeTUzLCAYEw@mail.gmail.com>
	<CADk+mPDshYyt4Q_2-Gds3KakAUEx642cacC__Th9JYixLr2DNw@mail.gmail.com>
	<51E3F3BB.7050606@quadrantsec.com>
	<51E41038.5030906@quadrantsec.com>
	<CADk+mPCmYQFbo_x3RG7C+EFF3b+0cVUXeTsk6a4=S6LYqw+JdQ@mail.gmail.com>
	<51E4481B.7000304@quadrantsec.com>
	<CADk+mPAsRV-uPBwtriFX13EoDpsdHkWUMT-Fuaw0YSieZdv-cA@mail.gmail.com>
	<51E4556D.3030706@quadrantsec.com>
	<CADk+mPBDcM-PQqpYsnczVuAgKYME0iTXMyctSHQrfzuqfKG0nA@mail.gmail.com>
Message-ID: <51E45722.8070508@quadrantsec.com>


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Not yet,  but that can be arranged :)

I literally just commented out all the 80% of it.   I'll throw it up and
let you know.


On 07/15/2013 04:06 PM, Rainer Gerhards wrote:
>
> I forgot: is your test code available online?
>
> Sent from phone, thus brief.
>
> Am 15.07.2013 22:02 schrieb "Champ Clark III" <cclark at quadrantsec.com
<mailto:cclark at quadrantsec.com>>:
>
>
> Sorry,  but same results :(   I'm using the same test code below but
with es_emptyStr(str) replaced with es_deleteStr(str)
>
>
>
> On 07/15/2013 03:46 PM, Rainer Gerhards wrote:
>
> > Use es_deleteStr instead of es_emptyStr. The latter just resets it
but does not free.  More explanations follow tomorrow.  Please report back.
>
> > Sent from phone, thus brief.
>
> > Am 15.07.2013 21:06 schrieb "Champ Clark III"
<cclark at quadrantsec.com <mailto:cclark at quadrantsec.com>
<mailto:cclark at quadrantsec.com> <mailto:cclark at quadrantsec.com>>:
>
>
>
> > Hello,
>
> > So - I've stripped down the code a good bit to see if I can't isolate
> > where I'm going wrong.  Below is what I got:
>
> > --<snip>--
> > str = es_newStrFromCStr(syslog_msg, strlen(syslog_msg));
> > ln_normalize(ctx, str, &lnevent);
>
> >        if(lnevent != NULL) {
> >                 es_emptyStr(str);
> >                 ee_fmtEventToRFC5424(lnevent, &str);
> >                 }
>
> > free(cstr);
> > es_deleteStr(str);
> > ee_deleteEvent(lnevent);
> > }
> > --<snip>--
>
> > It appears as soon as I add the "ee_fmtEventToRFC5424",  valgrind starts
> > to report the following:
>
> > ==21979== 69,872 bytes in 614 blocks are definitely lost in loss record
> > 52 of 54
> > ==21979==    at 0x4C2B6CD: malloc (in
> > /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
> > ==21979==    by 0x5457CD9: es_newStr (string.c:105)
> > ==21979==    by 0x5457D0E: es_newStrFromCStr (string.c:125)
> > ==21979==    by 0x40C167: sagan_normalize_liblognorm
> > (sagan-liblognorm.c:103)
> > ==21979==    by 0x41427F: Sagan_Blacklist (sagan-blacklist.c:167)
> > ==21979==    by 0x40BC07: Sagan_Processor (sagan-processor.c:123)
> > ==21979==    by 0x595EE99: start_thread (pthread_create.c:308)
>
> > If I remove the line,  that goes away.    Any thoughts?
>
> > Thanks for your time.
>
>
> >     _______________________________________________
> >     Lognorm mailing list
> >     Lognorm at lists.adiscon.com <mailto:Lognorm at lists.adiscon.com>
<mailto:Lognorm at lists.adiscon.com> <mailto:Lognorm at lists.adiscon.com>
> >     http://lists.adiscon.net/mailman/listinfo/lognorm
>
>
>
> > _______________________________________________
> > Lognorm mailing list
> > Lognorm at lists.adiscon.com <mailto:Lognorm at lists.adiscon.com>
> > http://lists.adiscon.net/mailman/listinfo/lognorm
>
>
>
>     _______________________________________________
>     Lognorm mailing list
>     Lognorm at lists.adiscon.com <mailto:Lognorm at lists.adiscon.com>
>     http://lists.adiscon.net/mailman/listinfo/lognorm
>
>
>
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm

- -- 
- - Quadrant Information Security
  Champ Clark III
  o: 800.538.9357 x 101
  c: 850.443.2440
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJR5FciAAoJENnmXt7Lmc3KVHAH/3+MA7bDO/ejGiOU/2WX24fl
3IXnZh/hmBFknXsiXQwWVy4X1Un/tQUJqYHtQ/TE+r3A7pp16WmJRhz0CzW9ySME
oJof02TXJZgiweFTOg3+JdK5JwWBdU7iPXzvCOB/IKbtsEladMcYCxYkPlaYWNA1
iQ37ZcHaUBej1FvW+qkCl0EMtcUdfolhcK2+NJMtPBxrxwfRDRcBRXDXuz2SzizE
EvJ1pMelkRRLhi5UIGONYQkFMF8FBGxb8tBp4iCLSRAzQHuoSuhpUHOhT6NzaVIg
GtQFscsYBHFDhl52E6Y3Rdv6/FZ2/C0yjrlHbGWACJPtvWLk47jsDiQxNQHodIw=
=+1FU
-----END PGP SIGNATURE-----

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20130715/515283dd/attachment.htm>

From rgerhards at hq.adiscon.com  Mon Jul 15 22:17:08 2013
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 15 Jul 2013 22:17:08 +0200
Subject: [Lognorm] Memory Leak in liblognorm?
In-Reply-To: <51E45722.8070508@quadrantsec.com>
References: <51E06051.5040901@quadrantsec.com>
	<CADk+mPDYAQ_4KvU26W9FTogp4NAW8vbYbWguXrpi7yzxAW3pjw@mail.gmail.com>
	<CADk+mPAeLW9rTeLOjheK47wgWCZHF7AzKbyhL-bdcrC8B_Xa6w@mail.gmail.com>
	<51E18A8C.1070502@quadrantsec.com>
	<CADk+mPCCiqrZ0w2ML4PTTa9wOiz3Dg-X6dCffaVO0xYriW-epA@mail.gmail.com>
	<51E1C572.9080003@quadrantsec.com>
	<0f4179f0-c6fc-49d4-86d1-7c835c1c3496@email.android.com>
	<CADk+mPB3AXgQedc4a2tB4yYrKOOLyaZT6YcON6ASM4MwSyiy+A@mail.gmail.com>
	<51E1E508.8040907@quadrantsec.com>
	<CADk+mPD04nm+pHb3h4nH=qgZVojH+s-Xq=xma+kTeTUzLCAYEw@mail.gmail.com>
	<CADk+mPDshYyt4Q_2-Gds3KakAUEx642cacC__Th9JYixLr2DNw@mail.gmail.com>
	<51E3F3BB.7050606@quadrantsec.com>
	<51E41038.5030906@quadrantsec.com>
	<CADk+mPCmYQFbo_x3RG7C+EFF3b+0cVUXeTsk6a4=S6LYqw+JdQ@mail.gmail.com>
	<51E4481B.7000304@quadrantsec.com>
	<CADk+mPAsRV-uPBwtriFX13EoDpsdHkWUMT-Fuaw0YSieZdv-cA@mail.gmail.com>
	<51E4556D.3030706@quadrantsec.com>
	<CADk+mPBDcM-PQqpYsnczVuAgKYME0iTXMyctSHQrfzuqfKG0nA@mail.gmail.com>
	<51E45722.8070508@quadrantsec.com>
Message-ID: <CADk+mPCzeD+w2yHj-_wA9u7iRxbE048JkDG22jOt1+3+pc=QRA@mail.gmail.com>

The fmt function creates a new string:
http://git.adiscon.com/?p=libee.git;a=blob;f=src/syslog_enc.c;h=d559521581242ec73a5687aae1de83ba4f2fbbe3;hb=HEAD#l150

So i think i need to see the full test code.

Sent from phone, thus brief.
Am 15.07.2013 22:10 schrieb "Champ Clark III" <cclark at quadrantsec.com>:

>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Not yet,  but that can be arranged :)
>
> I literally just commented out all the 80% of it.   I'll throw it up and
> let you know.
>
>
> On 07/15/2013 04:06 PM, Rainer Gerhards wrote:
> >
> > I forgot: is your test code available online?
> >
> > Sent from phone, thus brief.
> >
> > Am 15.07.2013 22:02 schrieb "Champ Clark III" <cclark at quadrantsec.com
> <mailto:cclark at quadrantsec.com> <cclark at quadrantsec.com>>:
> >
> >
> > Sorry,  but same results :(   I'm using the same test code below but
> with es_emptyStr(str) replaced with es_deleteStr(str)
> >
> >
> >
> > On 07/15/2013 03:46 PM, Rainer Gerhards wrote:
> >
> > > Use es_deleteStr instead of es_emptyStr. The latter just resets it but
> does not free.  More explanations follow tomorrow.  Please report back.
> >
> > > Sent from phone, thus brief.
> >
> > > Am 15.07.2013 21:06 schrieb "Champ Clark III" <cclark at quadrantsec.com
> <mailto:cclark at quadrantsec.com> <cclark at quadrantsec.com>
> <mailto:cclark at quadrantsec.com> <cclark at quadrantsec.com>
> <mailto:cclark at quadrantsec.com> <cclark at quadrantsec.com>>:
> >
> >
> >
> > > Hello,
> >
> > > So - I've stripped down the code a good bit to see if I can't isolate
> > > where I'm going wrong.  Below is what I got:
> >
> > > --<snip>--
> > > str = es_newStrFromCStr(syslog_msg, strlen(syslog_msg));
> > > ln_normalize(ctx, str, &lnevent);
> >
> > >        if(lnevent != NULL) {
> > >                 es_emptyStr(str);
> > >                 ee_fmtEventToRFC5424(lnevent, &str);
> > >                 }
> >
> > > free(cstr);
> > > es_deleteStr(str);
> > > ee_deleteEvent(lnevent);
> > > }
> > > --<snip>--
> >
> > > It appears as soon as I add the "ee_fmtEventToRFC5424",  valgrind
> starts
> > > to report the following:
> >
> > > ==21979== 69,872 bytes in 614 blocks are definitely lost in loss record
> > > 52 of 54
> > > ==21979==    at 0x4C2B6CD: malloc (in
> > > /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
> > > ==21979==    by 0x5457CD9: es_newStr (string.c:105)
> > > ==21979==    by 0x5457D0E: es_newStrFromCStr (string.c:125)
> > > ==21979==    by 0x40C167: sagan_normalize_liblognorm
> > > (sagan-liblognorm.c:103)
> > > ==21979==    by 0x41427F: Sagan_Blacklist (sagan-blacklist.c:167)
> > > ==21979==    by 0x40BC07: Sagan_Processor (sagan-processor.c:123)
> > > ==21979==    by 0x595EE99: start_thread (pthread_create.c:308)
> >
> > > If I remove the line,  that goes away.    Any thoughts?
> >
> > > Thanks for your time.
> >
> >
> > >     _______________________________________________
> > >     Lognorm mailing list
> > >     Lognorm at lists.adiscon.com <mailto:Lognorm at lists.adiscon.com><Lognorm at lists.adiscon.com>
> <mailto:Lognorm at lists.adiscon.com> <Lognorm at lists.adiscon.com>
> <mailto:Lognorm at lists.adiscon.com> <Lognorm at lists.adiscon.com>
> > >     http://lists.adiscon.net/mailman/listinfo/lognorm
> >
> >
> >
> > > _______________________________________________
> > > Lognorm mailing list
> > > Lognorm at lists.adiscon.com <mailto:Lognorm at lists.adiscon.com><Lognorm at lists.adiscon.com>
> > > http://lists.adiscon.net/mailman/listinfo/lognorm
> >
> >
> >
> >     _______________________________________________
> >     Lognorm mailing list
> >     Lognorm at lists.adiscon.com <mailto:Lognorm at lists.adiscon.com><Lognorm at lists.adiscon.com>
> >     http://lists.adiscon.net/mailman/listinfo/lognorm
> >
> >
> >
> > _______________________________________________
> > Lognorm mailing list
> > Lognorm at lists.adiscon.com
> > http://lists.adiscon.net/mailman/listinfo/lognorm
>
> - --
> - - Quadrant Information Security
>   Champ Clark III
>   o: 800.538.9357 x 101
>   c: 850.443.2440
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (GNU/Linux)
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQEcBAEBAgAGBQJR5FciAAoJENnmXt7Lmc3KVHAH/3+MA7bDO/ejGiOU/2WX24fl
> 3IXnZh/hmBFknXsiXQwWVy4X1Un/tQUJqYHtQ/TE+r3A7pp16WmJRhz0CzW9ySME
> oJof02TXJZgiweFTOg3+JdK5JwWBdU7iPXzvCOB/IKbtsEladMcYCxYkPlaYWNA1
> iQ37ZcHaUBej1FvW+qkCl0EMtcUdfolhcK2+NJMtPBxrxwfRDRcBRXDXuz2SzizE
> EvJ1pMelkRRLhi5UIGONYQkFMF8FBGxb8tBp4iCLSRAzQHuoSuhpUHOhT6NzaVIg
> GtQFscsYBHFDhl52E6Y3Rdv6/FZ2/C0yjrlHbGWACJPtvWLk47jsDiQxNQHodIw=
> =+1FU
> -----END PGP SIGNATURE-----
>
>
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20130715/e45653ee/attachment-0001.htm>

From cclark at quadrantsec.com  Mon Jul 15 22:19:30 2013
From: cclark at quadrantsec.com (Champ Clark III)
Date: Mon, 15 Jul 2013 16:19:30 -0400
Subject: [Lognorm] Memory Leak in liblognorm?
In-Reply-To: <CADk+mPBDcM-PQqpYsnczVuAgKYME0iTXMyctSHQrfzuqfKG0nA@mail.gmail.com>
References: <51E06051.5040901@quadrantsec.com>
	<CADk+mPDYAQ_4KvU26W9FTogp4NAW8vbYbWguXrpi7yzxAW3pjw@mail.gmail.com>
	<CADk+mPAeLW9rTeLOjheK47wgWCZHF7AzKbyhL-bdcrC8B_Xa6w@mail.gmail.com>
	<51E18A8C.1070502@quadrantsec.com>
	<CADk+mPCCiqrZ0w2ML4PTTa9wOiz3Dg-X6dCffaVO0xYriW-epA@mail.gmail.com>
	<51E1C572.9080003@quadrantsec.com>
	<0f4179f0-c6fc-49d4-86d1-7c835c1c3496@email.android.com>
	<CADk+mPB3AXgQedc4a2tB4yYrKOOLyaZT6YcON6ASM4MwSyiy+A@mail.gmail.com>
	<51E1E508.8040907@quadrantsec.com>
	<CADk+mPD04nm+pHb3h4nH=qgZVojH+s-Xq=xma+kTeTUzLCAYEw@mail.gmail.com>
	<CADk+mPDshYyt4Q_2-Gds3KakAUEx642cacC__Th9JYixLr2DNw@mail.gmail.com>
	<51E3F3BB.7050606@quadrantsec.com>
	<51E41038.5030906@quadrantsec.com>
	<CADk+mPCmYQFbo_x3RG7C+EFF3b+0cVUXeTsk6a4=S6LYqw+JdQ@mail.gmail.com>
	<51E4481B.7000304@quadrantsec.com>
	<CADk+mPAsRV-uPBwtriFX13EoDpsdHkWUMT-Fuaw0YSieZdv-cA@mail.gmail.com>
	<51E4556D.3030706@quadrantsec.com>
	<CADk+mPBDcM-PQqpYsnczVuAgKYME0iTXMyctSHQrfzuqfKG0nA@mail.gmail.com>
Message-ID: <51E45952.1060708@quadrantsec.com>


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

It's at:

https://github.com/beave/sagan/blob/master/src/sagan-liblognorm.c

See the function "sagan_normalize_liblognorm" (line 93).

There's not a lot to it (right now).    When my blacklist function calls
this as it is now,  the memory grows and grows.
If I disable the call to liblognorm,  it stays consistent.



On 07/15/2013 04:06 PM, Rainer Gerhards wrote:
>
> I forgot: is your test code available online?
>
> Sent from phone, thus brief.
>
> Am 15.07.2013 22:02 schrieb "Champ Clark III" <cclark at quadrantsec.com
<mailto:cclark at quadrantsec.com>>:
>
>
> Sorry,  but same results :(   I'm using the same test code below but
with es_emptyStr(str) replaced with es_deleteStr(str)
>
>
>
> On 07/15/2013 03:46 PM, Rainer Gerhards wrote:
>
> > Use es_deleteStr instead of es_emptyStr. The latter just resets it
but does not free.  More explanations follow tomorrow.  Please report back.
>
> > Sent from phone, thus brief.
>
> > Am 15.07.2013 21:06 schrieb "Champ Clark III"
<cclark at quadrantsec.com <mailto:cclark at quadrantsec.com>
<mailto:cclark at quadrantsec.com> <mailto:cclark at quadrantsec.com>>:
>
>
>
> > Hello,
>
> > So - I've stripped down the code a good bit to see if I can't isolate
> > where I'm going wrong.  Below is what I got:
>
> > --<snip>--
> > str = es_newStrFromCStr(syslog_msg, strlen(syslog_msg));
> > ln_normalize(ctx, str, &lnevent);
>
> >        if(lnevent != NULL) {
> >                 es_emptyStr(str);
> >                 ee_fmtEventToRFC5424(lnevent, &str);
> >                 }
>
> > free(cstr);
> > es_deleteStr(str);
> > ee_deleteEvent(lnevent);
> > }
> > --<snip>--
>
> > It appears as soon as I add the "ee_fmtEventToRFC5424",  valgrind starts
> > to report the following:
>
> > ==21979== 69,872 bytes in 614 blocks are definitely lost in loss record
> > 52 of 54
> > ==21979==    at 0x4C2B6CD: malloc (in
> > /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
> > ==21979==    by 0x5457CD9: es_newStr (string.c:105)
> > ==21979==    by 0x5457D0E: es_newStrFromCStr (string.c:125)
> > ==21979==    by 0x40C167: sagan_normalize_liblognorm
> > (sagan-liblognorm.c:103)
> > ==21979==    by 0x41427F: Sagan_Blacklist (sagan-blacklist.c:167)
> > ==21979==    by 0x40BC07: Sagan_Processor (sagan-processor.c:123)
> > ==21979==    by 0x595EE99: start_thread (pthread_create.c:308)
>
> > If I remove the line,  that goes away.    Any thoughts?
>
> > Thanks for your time.
>
>
> >     _______________________________________________
> >     Lognorm mailing list
> >     Lognorm at lists.adiscon.com <mailto:Lognorm at lists.adiscon.com>
<mailto:Lognorm at lists.adiscon.com> <mailto:Lognorm at lists.adiscon.com>
> >     http://lists.adiscon.net/mailman/listinfo/lognorm
>
>
>
> > _______________________________________________
> > Lognorm mailing list
> > Lognorm at lists.adiscon.com <mailto:Lognorm at lists.adiscon.com>
> > http://lists.adiscon.net/mailman/listinfo/lognorm
>
>
>
>     _______________________________________________
>     Lognorm mailing list
>     Lognorm at lists.adiscon.com <mailto:Lognorm at lists.adiscon.com>
>     http://lists.adiscon.net/mailman/listinfo/lognorm
>
>
>
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm

- -- 
- - Quadrant Information Security
  Champ Clark III
  o: 800.538.9357 x 101
  c: 850.443.2440
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJR5FlSAAoJENnmXt7Lmc3K/KkIAINRfPifLOsXVvdf8puDMMjH
MIls2b8T6R73IUmtZA7+1yO3BtRQKAx50/yBofvXX3uc6v3TskzezjKDIkdCuQJv
JieWERxsU7FcxoSfRPQT6QBEA6BGjKubwTPn7wwVBIhw5FfGkqMYTFfhcWoUovh5
SnO5dzRcLQ1w2RpiajFFBFRfkPEwjpPgVut0LZLTMMBx+v1mHZTFROnA9o/b43Jb
JSIjJRR6jPZYktGBhJZzvxfFB5FC9EX8n/gekhTBowC6nvJjVw1cg5CWyfzyIJ05
ml7gsSizU7sAvBp14ByTuSHvcbgkuGmRr7923pdFGR0z9xbk11P7Hm/9i/LjHHg=
=lZ/j
-----END PGP SIGNATURE-----

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20130715/948018cd/attachment.htm>

From rgerhards at hq.adiscon.com  Mon Jul 15 22:23:16 2013
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 15 Jul 2013 22:23:16 +0200
Subject: [Lognorm] Memory Leak in liblognorm?
In-Reply-To: <51E45952.1060708@quadrantsec.com>
References: <51E06051.5040901@quadrantsec.com>
	<CADk+mPDYAQ_4KvU26W9FTogp4NAW8vbYbWguXrpi7yzxAW3pjw@mail.gmail.com>
	<CADk+mPAeLW9rTeLOjheK47wgWCZHF7AzKbyhL-bdcrC8B_Xa6w@mail.gmail.com>
	<51E18A8C.1070502@quadrantsec.com>
	<CADk+mPCCiqrZ0w2ML4PTTa9wOiz3Dg-X6dCffaVO0xYriW-epA@mail.gmail.com>
	<51E1C572.9080003@quadrantsec.com>
	<0f4179f0-c6fc-49d4-86d1-7c835c1c3496@email.android.com>
	<CADk+mPB3AXgQedc4a2tB4yYrKOOLyaZT6YcON6ASM4MwSyiy+A@mail.gmail.com>
	<51E1E508.8040907@quadrantsec.com>
	<CADk+mPD04nm+pHb3h4nH=qgZVojH+s-Xq=xma+kTeTUzLCAYEw@mail.gmail.com>
	<CADk+mPDshYyt4Q_2-Gds3KakAUEx642cacC__Th9JYixLr2DNw@mail.gmail.com>
	<51E3F3BB.7050606@quadrantsec.com>
	<51E41038.5030906@quadrantsec.com>
	<CADk+mPCmYQFbo_x3RG7C+EFF3b+0cVUXeTsk6a4=S6LYqw+JdQ@mail.gmail.com>
	<51E4481B.7000304@quadrantsec.com>
	<CADk+mPAsRV-uPBwtriFX13EoDpsdHkWUMT-Fuaw0YSieZdv-cA@mail.gmail.com>
	<51E4556D.3030706@quadrantsec.com>
	<CADk+mPBDcM-PQqpYsnczVuAgKYME0iTXMyctSHQrfzuqfKG0nA@mail.gmail.com>
	<51E45952.1060708@quadrantsec.com>
Message-ID: <CADk+mPBNwXqCK9-wkaBpY3GCbU2OM0rEYmZfBD0D0MwQ6TyZ1A@mail.gmail.com>

Ln 109 add es_deleteStr

Sent from phone, thus brief.
Am 15.07.2013 22:19 schrieb "Champ Clark III" <cclark at quadrantsec.com>:

>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> It's at:
>
> https://github.com/beave/sagan/blob/master/src/sagan-liblognorm.c
>
> See the function "sagan_normalize_liblognorm" (line 93).
>
> There's not a lot to it (right now).    When my blacklist function calls
> this as it is now,  the memory grows and grows.
> If I disable the call to liblognorm,  it stays consistent.
>
>
>
> On 07/15/2013 04:06 PM, Rainer Gerhards wrote:
> >
> > I forgot: is your test code available online?
> >
> > Sent from phone, thus brief.
> >
> > Am 15.07.2013 22:02 schrieb "Champ Clark III" <cclark at quadrantsec.com
> <mailto:cclark at quadrantsec.com> <cclark at quadrantsec.com>>:
> >
> >
> > Sorry,  but same results :(   I'm using the same test code below but
> with es_emptyStr(str) replaced with es_deleteStr(str)
> >
> >
> >
> > On 07/15/2013 03:46 PM, Rainer Gerhards wrote:
> >
> > > Use es_deleteStr instead of es_emptyStr. The latter just resets it but
> does not free.  More explanations follow tomorrow.  Please report back.
> >
> > > Sent from phone, thus brief.
> >
> > > Am 15.07.2013 21:06 schrieb "Champ Clark III" <cclark at quadrantsec.com
> <mailto:cclark at quadrantsec.com> <cclark at quadrantsec.com>
> <mailto:cclark at quadrantsec.com> <cclark at quadrantsec.com>
> <mailto:cclark at quadrantsec.com> <cclark at quadrantsec.com>>:
> >
> >
> >
> > > Hello,
> >
> > > So - I've stripped down the code a good bit to see if I can't isolate
> > > where I'm going wrong.  Below is what I got:
> >
> > > --<snip>--
> > > str = es_newStrFromCStr(syslog_msg, strlen(syslog_msg));
> > > ln_normalize(ctx, str, &lnevent);
> >
> > >        if(lnevent != NULL) {
> > >                 es_emptyStr(str);
> > >                 ee_fmtEventToRFC5424(lnevent, &str);
> > >                 }
> >
> > > free(cstr);
> > > es_deleteStr(str);
> > > ee_deleteEvent(lnevent);
> > > }
> > > --<snip>--
> >
> > > It appears as soon as I add the "ee_fmtEventToRFC5424",  valgrind
> starts
> > > to report the following:
> >
> > > ==21979== 69,872 bytes in 614 blocks are definitely lost in loss record
> > > 52 of 54
> > > ==21979==    at 0x4C2B6CD: malloc (in
> > > /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
> > > ==21979==    by 0x5457CD9: es_newStr (string.c:105)
> > > ==21979==    by 0x5457D0E: es_newStrFromCStr (string.c:125)
> > > ==21979==    by 0x40C167: sagan_normalize_liblognorm
> > > (sagan-liblognorm.c:103)
> > > ==21979==    by 0x41427F: Sagan_Blacklist (sagan-blacklist.c:167)
> > > ==21979==    by 0x40BC07: Sagan_Processor (sagan-processor.c:123)
> > > ==21979==    by 0x595EE99: start_thread (pthread_create.c:308)
> >
> > > If I remove the line,  that goes away.    Any thoughts?
> >
> > > Thanks for your time.
> >
> >
> > >     _______________________________________________
> > >     Lognorm mailing list
> > >     Lognorm at lists.adiscon.com <mailto:Lognorm at lists.adiscon.com><Lognorm at lists.adiscon.com>
> <mailto:Lognorm at lists.adiscon.com> <Lognorm at lists.adiscon.com>
> <mailto:Lognorm at lists.adiscon.com> <Lognorm at lists.adiscon.com>
> > >     http://lists.adiscon.net/mailman/listinfo/lognorm
> >
> >
> >
> > > _______________________________________________
> > > Lognorm mailing list
> > > Lognorm at lists.adiscon.com <mailto:Lognorm at lists.adiscon.com><Lognorm at lists.adiscon.com>
> > > http://lists.adiscon.net/mailman/listinfo/lognorm
> >
> >
> >
> >     _______________________________________________
> >     Lognorm mailing list
> >     Lognorm at lists.adiscon.com <mailto:Lognorm at lists.adiscon.com><Lognorm at lists.adiscon.com>
> >     http://lists.adiscon.net/mailman/listinfo/lognorm
> >
> >
> >
> > _______________________________________________
> > Lognorm mailing list
> > Lognorm at lists.adiscon.com
> > http://lists.adiscon.net/mailman/listinfo/lognorm
>
> - --
> - - Quadrant Information Security
>   Champ Clark III
>   o: 800.538.9357 x 101
>   c: 850.443.2440
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (GNU/Linux)
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQEcBAEBAgAGBQJR5FlSAAoJENnmXt7Lmc3K/KkIAINRfPifLOsXVvdf8puDMMjH
> MIls2b8T6R73IUmtZA7+1yO3BtRQKAx50/yBofvXX3uc6v3TskzezjKDIkdCuQJv
> JieWERxsU7FcxoSfRPQT6QBEA6BGjKubwTPn7wwVBIhw5FfGkqMYTFfhcWoUovh5
> SnO5dzRcLQ1w2RpiajFFBFRfkPEwjpPgVut0LZLTMMBx+v1mHZTFROnA9o/b43Jb
> JSIjJRR6jPZYktGBhJZzvxfFB5FC9EX8n/gekhTBowC6nvJjVw1cg5CWyfzyIJ05
> ml7gsSizU7sAvBp14ByTuSHvcbgkuGmRr7923pdFGR0z9xbk11P7Hm/9i/LjHHg=
> =lZ/j
> -----END PGP SIGNATURE-----
>
>
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20130715/dadbd555/attachment-0001.htm>

From cclark at quadrantsec.com  Mon Jul 15 22:41:27 2013
From: cclark at quadrantsec.com (Champ Clark III)
Date: Mon, 15 Jul 2013 16:41:27 -0400
Subject: [Lognorm] Memory Leak in liblognorm?
In-Reply-To: <CADk+mPBNwXqCK9-wkaBpY3GCbU2OM0rEYmZfBD0D0MwQ6TyZ1A@mail.gmail.com>
References: <51E06051.5040901@quadrantsec.com>
	<51E18A8C.1070502@quadrantsec.com>
	<CADk+mPCCiqrZ0w2ML4PTTa9wOiz3Dg-X6dCffaVO0xYriW-epA@mail.gmail.com>
	<51E1C572.9080003@quadrantsec.com>
	<0f4179f0-c6fc-49d4-86d1-7c835c1c3496@email.android.com>
	<CADk+mPB3AXgQedc4a2tB4yYrKOOLyaZT6YcON6ASM4MwSyiy+A@mail.gmail.com>
	<51E1E508.8040907@quadrantsec.com>
	<CADk+mPD04nm+pHb3h4nH=qgZVojH+s-Xq=xma+kTeTUzLCAYEw@mail.gmail.com>
	<CADk+mPDshYyt4Q_2-Gds3KakAUEx642cacC__Th9JYixLr2DNw@mail.gmail.com>
	<51E3F3BB.7050606@quadrantsec.com>
	<51E41038.5030906@quadrantsec.com>
	<CADk+mPCmYQFbo_x3RG7C+EFF3b+0cVUXeTsk6a4=S6LYqw+JdQ@mail.gmail.com>
	<51E4481B.7000304@quadrantsec.com>
	<CADk+mPAsRV-uPBwtriFX13EoDpsdHkWUMT-Fuaw0YSieZdv-cA@mail.gmail.com>
	<51E4556D.3030706@quadrantsec.com>
	<CADk+mPBDcM-PQqpYsnczVuAgKYME0iTXMyctSHQrfzuqfKG0nA@mail.gmail.com>
	<51E45952.1060708@quadrantsec.com>
	<CADk+mPBNwXqCK9-wkaBpY3GCbU2OM0rEYmZfBD0D0MwQ6TyZ1A@mail.gmail.com>
Message-ID: <51E45E77.6050903@quadrantsec.com>


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ok.. Tried that earlier, but let me make sure.  Will let you  know the
results ASAP.


On 07/15/2013 04:23 PM, Rainer Gerhards wrote:
>
> Ln 109 add es_deleteStr
>
> Sent from phone, thus brief.
>
> Am 15.07.2013 22:19 schrieb "Champ Clark III" <cclark at quadrantsec.com
<mailto:cclark at quadrantsec.com>>:
>
>
> It's at:
>
> https://github.com/beave/sagan/blob/master/src/sagan-liblognorm.c
>
> See the function "sagan_normalize_liblognorm" (line 93).
>
> There's not a lot to it (right now).    When my blacklist function
calls this as it is now,  the memory grows and grows.
> If I disable the call to liblognorm,  it stays consistent.
>
>
>
> On 07/15/2013 04:06 PM, Rainer Gerhards wrote:
>
> > I forgot: is your test code available online?
>
> > Sent from phone, thus brief.
>
> > Am 15.07.2013 22:02 schrieb "Champ Clark III"
<cclark at quadrantsec.com <mailto:cclark at quadrantsec.com>
<mailto:cclark at quadrantsec.com> <mailto:cclark at quadrantsec.com>>:
>
>
> > Sorry,  but same results :(   I'm using the same test code below but
with es_emptyStr(str) replaced with es_deleteStr(str)
>
>
>
> > On 07/15/2013 03:46 PM, Rainer Gerhards wrote:
>
> > > Use es_deleteStr instead of es_emptyStr. The latter just resets it
but does not free.  More explanations follow tomorrow.  Please report back.
>
> > > Sent from phone, thus brief.
>
> > > Am 15.07.2013 21:06 schrieb "Champ Clark III"
<cclark at quadrantsec.com <mailto:cclark at quadrantsec.com>
<mailto:cclark at quadrantsec.com> <mailto:cclark at quadrantsec.com>
<mailto:cclark at quadrantsec.com> <mailto:cclark at quadrantsec.com>
<mailto:cclark at quadrantsec.com> <mailto:cclark at quadrantsec.com>>:
>
>
>
> > > Hello,
>
> > > So - I've stripped down the code a good bit to see if I can't isolate
> > > where I'm going wrong.  Below is what I got:
>
> > > --<snip>--
> > > str = es_newStrFromCStr(syslog_msg, strlen(syslog_msg));
> > > ln_normalize(ctx, str, &lnevent);
>
> > >        if(lnevent != NULL) {
> > >                 es_emptyStr(str);
> > >                 ee_fmtEventToRFC5424(lnevent, &str);
> > >                 }
>
> > > free(cstr);
> > > es_deleteStr(str);
> > > ee_deleteEvent(lnevent);
> > > }
> > > --<snip>--
>
> > > It appears as soon as I add the "ee_fmtEventToRFC5424",  valgrind
starts
> > > to report the following:
>
> > > ==21979== 69,872 bytes in 614 blocks are definitely lost in loss
record
> > > 52 of 54
> > > ==21979==    at 0x4C2B6CD: malloc (in
> > > /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
> > > ==21979==    by 0x5457CD9: es_newStr (string.c:105)
> > > ==21979==    by 0x5457D0E: es_newStrFromCStr (string.c:125)
> > > ==21979==    by 0x40C167: sagan_normalize_liblognorm
> > > (sagan-liblognorm.c:103)
> > > ==21979==    by 0x41427F: Sagan_Blacklist (sagan-blacklist.c:167)
> > > ==21979==    by 0x40BC07: Sagan_Processor (sagan-processor.c:123)
> > > ==21979==    by 0x595EE99: start_thread (pthread_create.c:308)
>
> > > If I remove the line,  that goes away.    Any thoughts?
>
> > > Thanks for your time.
>
>
> > >     _______________________________________________
> > >     Lognorm mailing list
> > >     Lognorm at lists.adiscon.com <mailto:Lognorm at lists.adiscon.com>
<mailto:Lognorm at lists.adiscon.com> <mailto:Lognorm at lists.adiscon.com>
<mailto:Lognorm at lists.adiscon.com> <mailto:Lognorm at lists.adiscon.com>
<mailto:Lognorm at lists.adiscon.com> <mailto:Lognorm at lists.adiscon.com>
> > >     http://lists.adiscon.net/mailman/listinfo/lognorm
>
>
>
> > > _______________________________________________
> > > Lognorm mailing list
> > > Lognorm at lists.adiscon.com <mailto:Lognorm at lists.adiscon.com>
<mailto:Lognorm at lists.adiscon.com> <mailto:Lognorm at lists.adiscon.com>
> > > http://lists.adiscon.net/mailman/listinfo/lognorm
>
>
>
> >     _______________________________________________
> >     Lognorm mailing list
> >     Lognorm at lists.adiscon.com <mailto:Lognorm at lists.adiscon.com>
<mailto:Lognorm at lists.adiscon.com> <mailto:Lognorm at lists.adiscon.com>
> >     http://lists.adiscon.net/mailman/listinfo/lognorm
>
>
>
> > _______________________________________________
> > Lognorm mailing list
> > Lognorm at lists.adiscon.com <mailto:Lognorm at lists.adiscon.com>
> > http://lists.adiscon.net/mailman/listinfo/lognorm
>
>
>
>     _______________________________________________
>     Lognorm mailing list
>     Lognorm at lists.adiscon.com <mailto:Lognorm at lists.adiscon.com>
>     http://lists.adiscon.net/mailman/listinfo/lognorm
>
>
>
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm

- -- 
- - Quadrant Information Security
  Champ Clark III
  o: 800.538.9357 x 101
  c: 850.443.2440
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJR5F53AAoJENnmXt7Lmc3K2yAH/jiD2MRjALwF78X72PGObxnC
fqVjlydZ0z5u9wH6aDeFhfs8QVkx52pxOucMQootXA6AIAYyexr+A7RTQE1RaRxL
iWv68ZJOaCeRgTvnywZh9yozykEuhAa3FgwI2weXipWfnAjxPpoQvD/eSir4XAoy
B7OVqm8nvqa0jrRvTm9MZZeu712GqqMRPJm4VXutxzmZDmQrqaJ2KYnHIfvAwkv2
iCvAiHHbkH1yJSB7X8MbNxlCmig7loPC8hWLSiWD+1blcp5AtknpBdKp9HYEa/cs
uU4vQVx3t0tEzY1Nh8vQCKGpdPGMNUi6oHGycdKWJZAJI609GVWRNKAdYK5KTJE=
=1Vjd
-----END PGP SIGNATURE-----

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20130715/20926b15/attachment.htm>

From cclark at quadrantsec.com  Mon Jul 15 23:19:29 2013
From: cclark at quadrantsec.com (Champ Clark III)
Date: Mon, 15 Jul 2013 17:19:29 -0400
Subject: [Lognorm] Memory Leak in liblognorm?
In-Reply-To: <CADk+mPBNwXqCK9-wkaBpY3GCbU2OM0rEYmZfBD0D0MwQ6TyZ1A@mail.gmail.com>
References: <51E06051.5040901@quadrantsec.com>
	<51E18A8C.1070502@quadrantsec.com>
	<CADk+mPCCiqrZ0w2ML4PTTa9wOiz3Dg-X6dCffaVO0xYriW-epA@mail.gmail.com>
	<51E1C572.9080003@quadrantsec.com>
	<0f4179f0-c6fc-49d4-86d1-7c835c1c3496@email.android.com>
	<CADk+mPB3AXgQedc4a2tB4yYrKOOLyaZT6YcON6ASM4MwSyiy+A@mail.gmail.com>
	<51E1E508.8040907@quadrantsec.com>
	<CADk+mPD04nm+pHb3h4nH=qgZVojH+s-Xq=xma+kTeTUzLCAYEw@mail.gmail.com>
	<CADk+mPDshYyt4Q_2-Gds3KakAUEx642cacC__Th9JYixLr2DNw@mail.gmail.com>
	<51E3F3BB.7050606@quadrantsec.com>
	<51E41038.5030906@quadrantsec.com>
	<CADk+mPCmYQFbo_x3RG7C+EFF3b+0cVUXeTsk6a4=S6LYqw+JdQ@mail.gmail.com>
	<51E4481B.7000304@quadrantsec.com>
	<CADk+mPAsRV-uPBwtriFX13EoDpsdHkWUMT-Fuaw0YSieZdv-cA@mail.gmail.com>
	<51E4556D.3030706@quadrantsec.com>
	<CADk+mPBDcM-PQqpYsnczVuAgKYME0iTXMyctSHQrfzuqfKG0nA@mail.gmail.com>
	<51E45952.1060708@quadrantsec.com>
	<CADk+mPBNwXqCK9-wkaBpY3GCbU2OM0rEYmZfBD0D0MwQ6TyZ1A@mail.gmail.com>
Message-ID: <51E46761.1000602@quadrantsec.com>


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

So _far_ that seems to have done the trick.  I'm still testing,  so I
don't want to get to far out and jinx it :)

I'm a bit confused,  because now I have two es_deleteStr(str).  My
thought was that it would be similar to a double free,
but it's obviously not.   That is,  I added the es_deleteStr(str) at
line 109,  but the one at 188 is still there as well!

I've pulled back in the rest of the code that I had commented out and
I'm going to let it run a good bit longer on this
busy box.

Will let you know the results.



On 07/15/2013 04:23 PM, Rainer Gerhards wrote:
>
> Ln 109 add es_deleteStr
>
> Sent from phone, thus brief.
>
> Am 15.07.2013 22:19 schrieb "Champ Clark III" <cclark at quadrantsec.com
<mailto:cclark at quadrantsec.com>>:
>
=
- -- 
- - Quadrant Information Security
  Champ Clark III
  o: 800.538.9357 x 101
  c: 850.443.2440
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJR5GdgAAoJENnmXt7Lmc3KuHYH/1Y7vtf3aQjRBqsRBp4k5o8O
6hB1/8IYr6246CmLWKXWnpewHk0v2CezXdbzPZdETLTHULq1wtBZBFcgU674pgOP
tZ2t1tDL6PexCn3c0nNsBmEZoP2WAu79XqP2OzgeujlLLC+/YCVbz/jG6JPvg6tC
DURyTq8gNcs4OvNMzEtlkLeTK/k9J9fkCxg7La/rzcmUwZkRxDvpGcKY+ur0R5Zq
4xlMcf1gv51K3pS4OypbWZP5NfqFnZzCn4ePr3GVAhlmhUF6uhDn9Hc/vUzekUUh
X1CBURCGPbBJ0chAkJ4n4D16IyZhqS48ogxrgpKasKKgJAy58d7WS+Oz4Wr66l4=
=P6dm
-----END PGP SIGNATURE-----

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20130715/ae4ed652/attachment.htm>

From rgerhards at hq.adiscon.com  Tue Jul 16 08:18:41 2013
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 16 Jul 2013 08:18:41 +0200
Subject: [Lognorm] Memory Leak in liblognorm?
In-Reply-To: <51E46761.1000602@quadrantsec.com>
References: <51E06051.5040901@quadrantsec.com>
	<51E18A8C.1070502@quadrantsec.com>
	<CADk+mPCCiqrZ0w2ML4PTTa9wOiz3Dg-X6dCffaVO0xYriW-epA@mail.gmail.com>
	<51E1C572.9080003@quadrantsec.com>
	<0f4179f0-c6fc-49d4-86d1-7c835c1c3496@email.android.com>
	<CADk+mPB3AXgQedc4a2tB4yYrKOOLyaZT6YcON6ASM4MwSyiy+A@mail.gmail.com>
	<51E1E508.8040907@quadrantsec.com>
	<CADk+mPD04nm+pHb3h4nH=qgZVojH+s-Xq=xma+kTeTUzLCAYEw@mail.gmail.com>
	<CADk+mPDshYyt4Q_2-Gds3KakAUEx642cacC__Th9JYixLr2DNw@mail.gmail.com>
	<51E3F3BB.7050606@quadrantsec.com>
	<51E41038.5030906@quadrantsec.com>
	<CADk+mPCmYQFbo_x3RG7C+EFF3b+0cVUXeTsk6a4=S6LYqw+JdQ@mail.gmail.com>
	<51E4481B.7000304@quadrantsec.com>
	<CADk+mPAsRV-uPBwtriFX13EoDpsdHkWUMT-Fuaw0YSieZdv-cA@mail.gmail.com>
	<51E4556D.3030706@quadrantsec.com>
	<CADk+mPBDcM-PQqpYsnczVuAgKYME0iTXMyctSHQrfzuqfKG0nA@mail.gmail.com>
	<51E45952.1060708@quadrantsec.com>
	<CADk+mPBNwXqCK9-wkaBpY3GCbU2OM0rEYmZfBD0D0MwQ6TyZ1A@mail.gmail.com>
	<51E46761.1000602@quadrantsec.com>
Message-ID: <CADk+mPDSjzNr+Anur84pVDFNCdpvZArGvK1w3be77EFZweo-Vg@mail.gmail.com>

On Mon, Jul 15, 2013 at 11:19 PM, Champ Clark III <cclark at quadrantsec.com>wrote:

>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> So _far_ that seems to have done the trick.  I'm still testing,  so I
> don't want to get to far out and jinx it :)
>
>
believe me, that's the fix ;)

The core problem is probably that the API doc for that set of functions is
virtually non-existent. I reviewed the situation and that part of the lib
was written when CEE began to change object model and consequently the API
would have been needed to change. That was when I stopped working on that
part looking for what would come up. As we all now know, actually nothing
came up later on and so we sticked with the state we currently have. :-( I
spoke about changing the API to be more JSONish a couple of month ago. I
should really do that and remove the CEE dependencies.

Anyhow, on the solution:

I'm a bit confused,  because now I have two es_deleteStr(str).  My thought
> was that it would be similar to a double free,
> but it's obviously not.   That is,  I added the es_deleteStr(str) at line
> 109,  but the one at 188 is still there as well!
>
>
The key point to understand is that ee_fmtEventToRFC5424 returns a *new*
estr into the result pointer you provide (str in your case). Have a look
here, and carefully look at line 156, where it is created:

http://git.adiscon.com/?p=libee.git;a=blob;f=src/syslog_enc.c;h=d559521581242ec73a5687aae1de83ba4f2fbbe3;hb=HEAD#l150

So while you have just one variable, this one variable points to different
objects at different points in time. So when you assign a new object to it
(like you do with ee_fmtEventToRFC5424), you need to free the previous
object, as you otherwise can never do that.

Coming back to my malloc example, this is absolutely equivalent to this
code sequence:

char *str;
str = malloc(100);
str = strdup("a");
str = strdup("b");
free(str);

This sequence will leak 102+ bytes of memory on each iteration: one for the
100-byte malloc block, one for the two byte string "a" (and the + is for
the malloc memory block headers, so it will definitely leak more than 102
bytes). Correctly, this is done as follows:

char *str;
str = malloc(100);
free(str);
str = strdup("a");
free(str);
str = strdup("b");
free(str);

Here, you also supply the same variable to free, but each time it points to
a different memory block. So no double-free.

It's exactly the same with libee functions that return strings (they do
return new strings because the idea is that the caller would store and use
them for further processing - I agree that's questionable).

I've pulled back in the rest of the code that I had commented out and I'm
> going to let it run a good bit longer on this
> busy box.
>
>
You must make sure that you consistently free the pervious object if you
re-use a variable to point to a new object (or alternatively have a single
variable for each object - one that is not being reused).

As a side-note: things like the almost constant property name I tend to
store in global variables and only create once and use the same set of
variables all the time without re-creating them. Depending on what you do,
this can make some performace diffrence.

I hope this clarifies now. Let me know any questions you may still have.

Rainer

> Will let you know the results.
>
>
>
>
> On 07/15/2013 04:23 PM, Rainer Gerhards wrote:
> >
> > Ln 109 add es_deleteStr
> >
> > Sent from phone, thus brief.
> >
> > Am 15.07.2013 22:19 schrieb "Champ Clark III" <cclark at quadrantsec.com
> <mailto:cclark at quadrantsec.com> <cclark at quadrantsec.com>>:
> >
>
> =
> - --
> - - Quadrant Information Security
>   Champ Clark III
>   o: 800.538.9357 x 101
>   c: 850.443.2440
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (GNU/Linux)
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQEcBAEBAgAGBQJR5GdgAAoJENnmXt7Lmc3KuHYH/1Y7vtf3aQjRBqsRBp4k5o8O
> 6hB1/8IYr6246CmLWKXWnpewHk0v2CezXdbzPZdETLTHULq1wtBZBFcgU674pgOP
> tZ2t1tDL6PexCn3c0nNsBmEZoP2WAu79XqP2OzgeujlLLC+/YCVbz/jG6JPvg6tC
> DURyTq8gNcs4OvNMzEtlkLeTK/k9J9fkCxg7La/rzcmUwZkRxDvpGcKY+ur0R5Zq
> 4xlMcf1gv51K3pS4OypbWZP5NfqFnZzCn4ePr3GVAhlmhUF6uhDn9Hc/vUzekUUh
> X1CBURCGPbBJ0chAkJ4n4D16IyZhqS48ogxrgpKasKKgJAy58d7WS+Oz4Wr66l4=
> =P6dm
> -----END PGP SIGNATURE-----
>
>
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20130716/262f2fa6/attachment-0001.htm>

From cclark at quadrantsec.com  Tue Jul 16 17:24:28 2013
From: cclark at quadrantsec.com (Champ Clark III)
Date: Tue, 16 Jul 2013 11:24:28 -0400
Subject: [Lognorm] Memory Leak in liblognorm?
In-Reply-To: <CADk+mPDSjzNr+Anur84pVDFNCdpvZArGvK1w3be77EFZweo-Vg@mail.gmail.com>
References: <51E06051.5040901@quadrantsec.com>
	<51E1C572.9080003@quadrantsec.com>
	<0f4179f0-c6fc-49d4-86d1-7c835c1c3496@email.android.com>
	<CADk+mPB3AXgQedc4a2tB4yYrKOOLyaZT6YcON6ASM4MwSyiy+A@mail.gmail.com>
	<51E1E508.8040907@quadrantsec.com>
	<CADk+mPD04nm+pHb3h4nH=qgZVojH+s-Xq=xma+kTeTUzLCAYEw@mail.gmail.com>
	<CADk+mPDshYyt4Q_2-Gds3KakAUEx642cacC__Th9JYixLr2DNw@mail.gmail.com>
	<51E3F3BB.7050606@quadrantsec.com>
	<51E41038.5030906@quadrantsec.com>
	<CADk+mPCmYQFbo_x3RG7C+EFF3b+0cVUXeTsk6a4=S6LYqw+JdQ@mail.gmail.com>
	<51E4481B.7000304@quadrantsec.com>
	<CADk+mPAsRV-uPBwtriFX13EoDpsdHkWUMT-Fuaw0YSieZdv-cA@mail.gmail.com>
	<51E4556D.3030706@quadrantsec.com>
	<CADk+mPBDcM-PQqpYsnczVuAgKYME0iTXMyctSHQrfzuqfKG0nA@mail.gmail.com>
	<51E45952.1060708@quadrantsec.com>
	<CADk+mPBNwXqCK9-wkaBpY3GCbU2OM0rEYmZfBD0D0MwQ6TyZ1A@mail.gmail.com>
	<51E46761.1000602@quadrantsec.com>
	<CADk+mPDSjzNr+Anur84pVDFNCdpvZArGvK1w3be77EFZweo-Vg@mail.gmail.com>
Message-ID: <51E565AC.2060707@quadrantsec.com>


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


> > believe me, that's the fix ;)

.. Yep :)

> > The key point to understand is that ee_fmtEventToRFC5424 returns a *new* estr into the
result pointer you provide (str in your case). Have a look here, and
carefully look at line 156, where it is created:
> >
http://git.adiscon.com/?p=libee.git;a=blob;f=src/syslog_enc.c;h=d559521581242ec73a5687aae1de83ba4f2fbbe3;hb=HEAD#l150
> > So while you have just one variable, this one variable points to
different objects at different points in time. So when you assign a new
object to it (like you do with ee_fmtEventToRFC5424), you need to free
the previous object, as you otherwise can never do that.

..snip..
>
> > Here, you also supply the same variable to free, but each time it
points to a different memory block. So no double-free.

Ahhh!  Okay!  Thank you for the explanation,  that really clears it up!

> > You must make sure that you consistently free the pervious object if you re-use a variable to
point to a new object (or alternatively have a single variable for each
object - one that is not being reused).
>
> > As a side-note: things like the almost constant property name I tend
to store in global variables and only create once and use the same set
of variables all the time without re-creating them. Depending on what
you do, this can make some performace diffrence.
>

Thanks for the tip,  I'll play with it.

Again.. Thank you for the help!  I really appreciate it!



- -- 
- - Quadrant Information Security
  Champ Clark III
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iQEcBAEBAgAGBQJR5WWsAAoJENnmXt7Lmc3K57cH/jS3nCvg3BeGdvgaZv4RV3PX
c5Ds71DRaGK0Jjxly3+M+1qWndAjCDcxCYZSnmNZzkuTFnNJ0eDV2pkSAKqo87f4
vde06oGemOIzAgUZAfSpj3ssXbpL2n8F/KAS3x8DTGwOWybtgSQxWUrM/ds7xtAV
e0vepSlDjCDmr235OBwbKD4M+TYK2uD/rIGe/AwyyXk5djv8CjNZUZWwzgRha8q/
aNG3M3CsqnYXn2pZBD1zlXvhFYjp0x9KdqFy11CPY6nXyWAYwG0hXJR2vz9NPUbY
7Ax6sa+fqaxg2PkQSs30I4HerGEwipLYEgeDhqaiIanwFwZIw8c8R/rKpkb9++w=
=AI3L
-----END PGP SIGNATURE-----


From rgerhards at hq.adiscon.com  Tue Jul 16 17:51:16 2013
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Tue, 16 Jul 2013 17:51:16 +0200
Subject: [Lognorm] Memory Leak in liblognorm?
In-Reply-To: <51E565AC.2060707@quadrantsec.com>
References: <51E06051.5040901@quadrantsec.com>
	<51E1C572.9080003@quadrantsec.com>
	<0f4179f0-c6fc-49d4-86d1-7c835c1c3496@email.android.com>
	<CADk+mPB3AXgQedc4a2tB4yYrKOOLyaZT6YcON6ASM4MwSyiy+A@mail.gmail.com>
	<51E1E508.8040907@quadrantsec.com>
	<CADk+mPD04nm+pHb3h4nH=qgZVojH+s-Xq=xma+kTeTUzLCAYEw@mail.gmail.com>
	<CADk+mPDshYyt4Q_2-Gds3KakAUEx642cacC__Th9JYixLr2DNw@mail.gmail.com>
	<51E3F3BB.7050606@quadrantsec.com>
	<51E41038.5030906@quadrantsec.com>
	<CADk+mPCmYQFbo_x3RG7C+EFF3b+0cVUXeTsk6a4=S6LYqw+JdQ@mail.gmail.com>
	<51E4481B.7000304@quadrantsec.com>
	<CADk+mPAsRV-uPBwtriFX13EoDpsdHkWUMT-Fuaw0YSieZdv-cA@mail.gmail.com>
	<51E4556D.3030706@quadrantsec.com>
	<CADk+mPBDcM-PQqpYsnczVuAgKYME0iTXMyctSHQrfzuqfKG0nA@mail.gmail.com>
	<51E45952.1060708@quadrantsec.com>
	<CADk+mPBNwXqCK9-wkaBpY3GCbU2OM0rEYmZfBD0D0MwQ6TyZ1A@mail.gmail.com>
	<51E46761.1000602@quadrantsec.com>
	<CADk+mPDSjzNr+Anur84pVDFNCdpvZArGvK1w3be77EFZweo-Vg@mail.gmail.com>
	<51E565AC.2060707@quadrantsec.com>
Message-ID: <CADk+mPAzw6WqdW=dxyisDbRJ+7weFiEq925ZtYzJC7cCu4g1zQ@mail.gmail.com>

glad it helped -- and sorry for previous briefness ;)

Rainer


On Tue, Jul 16, 2013 at 5:24 PM, Champ Clark III <cclark at quadrantsec.com>wrote:

>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
>
> > > believe me, that's the fix ;)
>
> .. Yep :)
>
> > > The key point to understand is that ee_fmtEventToRFC5424 returns a
> *new* estr into the
> result pointer you provide (str in your case). Have a look here, and
> carefully look at line 156, where it is created:
> > >
>
> http://git.adiscon.com/?p=libee.git;a=blob;f=src/syslog_enc.c;h=d559521581242ec73a5687aae1de83ba4f2fbbe3;hb=HEAD#l150
> > > So while you have just one variable, this one variable points to
> different objects at different points in time. So when you assign a new
> object to it (like you do with ee_fmtEventToRFC5424), you need to free
> the previous object, as you otherwise can never do that.
>
> ..snip..
> >
> > > Here, you also supply the same variable to free, but each time it
> points to a different memory block. So no double-free.
>
> Ahhh!  Okay!  Thank you for the explanation,  that really clears it up!
>
> > > You must make sure that you consistently free the pervious object if
> you re-use a variable to
> point to a new object (or alternatively have a single variable for each
> object - one that is not being reused).
> >
> > > As a side-note: things like the almost constant property name I tend
> to store in global variables and only create once and use the same set
> of variables all the time without re-creating them. Depending on what
> you do, this can make some performace diffrence.
> >
>
> Thanks for the tip,  I'll play with it.
>
> Again.. Thank you for the help!  I really appreciate it!
>
>
>
> - --
> - - Quadrant Information Security
>   Champ Clark III
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (GNU/Linux)
> Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/
>
> iQEcBAEBAgAGBQJR5WWsAAoJENnmXt7Lmc3K57cH/jS3nCvg3BeGdvgaZv4RV3PX
> c5Ds71DRaGK0Jjxly3+M+1qWndAjCDcxCYZSnmNZzkuTFnNJ0eDV2pkSAKqo87f4
> vde06oGemOIzAgUZAfSpj3ssXbpL2n8F/KAS3x8DTGwOWybtgSQxWUrM/ds7xtAV
> e0vepSlDjCDmr235OBwbKD4M+TYK2uD/rIGe/AwyyXk5djv8CjNZUZWwzgRha8q/
> aNG3M3CsqnYXn2pZBD1zlXvhFYjp0x9KdqFy11CPY6nXyWAYwG0hXJR2vz9NPUbY
> 7Ax6sa+fqaxg2PkQSs30I4HerGEwipLYEgeDhqaiIanwFwZIw8c8R/rKpkb9++w=
> =AI3L
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20130716/fc2fddc7/attachment.htm>

From friedl at adiscon.com  Thu Jul 18 16:33:56 2013
From: friedl at adiscon.com (Florian Riedl)
Date: Thu, 18 Jul 2013 16:33:56 +0200
Subject: [Lognorm] Fwd: liblognorm 0.3.7 released
In-Reply-To: <CAAq4--SdM_PbB97FXKKOoL-W6bVF_QTROarpxzS-f=A_kaDNxQ@mail.gmail.com>
References: <CAAq4--S8yudNjJDEcAZorq_rcZzrNEyCcrGZZ+eR-yH8=8UowA@mail.gmail.com>
	<CAAq4--SdM_PbB97FXKKOoL-W6bVF_QTROarpxzS-f=A_kaDNxQ@mail.gmail.com>
Message-ID: <CAAq4--RBj-2J8j4eW-33PoVmvC8XkiTsSNKz3xVc97AZgru1sw@mail.gmail.com>

Hi all,

We have just released liblognorm 0.3.7. This release contains a single API
extension which is needed by John Hopper?s python bindings (and also
contributed by him ? thanks!)

Changes

Version 0.3.7, 2013-07-17

   - added support to load single samples
   Thanks to John Hopper for the patch

Download:
http://www.liblognorm.com/download/liblognorm-0-3-7/

As always, feedback is appreciated.

Best regards,
Florian Riedl
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20130718/8e17776c/attachment.htm>