[Lognorm] log classification with liblognorm

[Castillo, Jose Contractor]

Hello all,

Can anybody please tell me if (and how) rsyslog supports log classification with liblognorm (message classification tag, not the field tag)?

Thanks in advance.

Jose

-----Original Message-----
From: lognorm-bounces at lists.adiscon.com [mailto:lognorm-bounces at lists.adiscon.com] On Behalf Of David Lang
Sent: Wednesday, June 26, 2013 4:59 PM
To: lognorm
Subject: Re: [Lognorm] log classification with liblognorm

Ok, now I see what you are talking about. You are talking about the message classification tag, not the field tag. My mistake.

I don't know if and how rsyslog supports this.

David Lang

On Wed, 26 Jun 2013, David Lang wrote:

> First off, I have not used mmlognorm for anything other than seeing 
> that it works, so take that into consideration :-)
>
> However, as I'm looking at your rule, you have "mytag" outside of any 
> matching, so it would be treated as a literal, just like "Denied ICMP"
> wouldn't it?
>
> in your syslog template, try putting %$!number2% and see if it outputs 
> the correct value. it looks to me like that variable is being set correctly.
>
> David Lang
>
> On Wed, 26 Jun 2013, Castillo, Jose   Contractor wrote:
>
>> David,
>> 
>> I modified the template:
>> template(name="testFormat" type="string" 
>> string="%$!all-json%,tag='%$!mytag%'")
>> 
>> I also modified the rule to include mytag as a tag:
>> rule=mytag:%date:word% %host:ipv4% : 
>> %%ASA-%number0:number%-%number1:number%: Denied ICMP 
>> type=%number2:number%, code=%number3:number% from %origin:ipv4% on 
>> interface outside
>> 
>> And now I getting :
>> { "origin": "77.2.2.2", "number3": "0", "number2": "8", "number1": 
>> "313001", "number0": "3", "host": "0.0.0.0", "date": 
>> "2013-06-26T10:47:42+01:00" },tag=''
>> 
>> No tags are being written to the output file.
>> 
>> What am I doing wrong?
>> 
>> Thanks,
>> 
>> Jose
>> 
>> -----Original Message-----
>> From: lognorm-bounces at lists.adiscon.com 
>> [mailto:lognorm-bounces at lists.adiscon.com] On Behalf Of David Lang
>> Sent: Wednesday, June 26, 2013 3:11 PM
>> To: lognorm
>> Subject: Re: [Lognorm] log classification with liblognorm
>> 
>> the tags are accessed in rsyslog by $!<tag name>
>> 
>> so where you would have done %hostname% you could do %$!mytag%
>> 
>> David Lang
>> 
>> On Wed, 26 Jun 2013, Castillo, Jose   Contractor wrote:
>> 
>>> Date: Wed, 26 Jun 2013 16:00:06 -0400
>>> From: "Castillo, Jose   Contractor" 
>>> <Jose.Castillo at ssa.gov<mailto:Jose.Castillo at ssa.gov>>
>>> Reply-To: lognorm
>>> <lognorm at lists.adiscon.com<mailto:lognorm at lists.adiscon.com>>
>>> To: "lognorm at lists.adiscon.com<mailto:lognorm at lists.adiscon.com>" 
>>> <lognorm at lists.adiscon.com<mailto:lognorm at lists.adiscon.com>>
>>> Subject: [Lognorm] log classification with liblognorm
>>> 
>>> Hello all,
>>> 
>>> I started to play using rsyslog/liblognorm a week ago, I'm testing 
>>> them on a CentOS 6.4 virtual machine, and next packages have been 
>>> installed from Adiscon repository:
>>> rsyslog-mysql-7.4.1-1.el6.x86_64
>>> rsyslog-mmjsonparse-7.4.1-1.el6.x86_64
>>> rsyslog-7.4.1-1.el6.x86_64
>>> rsyslog-mmnormalize-7.4.1-1.el6.x86_64
>>> rsyslog-udpspoof-7.4.1-1.el6.x86_64
>>> rsyslog-elasticsearch-7.4.1-1.el6.x86_64
>>> 
>>> I was reading next link, and now I'm trying to select subsets of 
>>> messages based on liblognorm tags.
>>> http://blog.gerhards.net/2011/04/log-classification-with-liblognorm.
>>> ht
>>> ml
>>> 
>>> So far, I was able to parse a message using mmnormalize module, and 
>>> now I want to create outputs based on the liblognorm tag(s).
>>> Is this option supported within rsyslog.conf? Any example how to do it?
>>> 
>>> Below my testing files:
>>> 
>>> =================================================================
>>> rsyslog.conf file:
>>> module (load="imudp")
>>> module (load="mmnormalize")
>>> module (load="mmjsonparse")
>>> 
>>> input(type="imudp" address="192.168.1.1" port="514" ruleset="test")
>>> 
>>> template(name="testFormat" type="string" string="%$!all-json%\n")
>>> 
>>> ruleset(name="test") {
>>>        action(type="mmnormalize" userawmsg="on" 
>>> rulebase="/data/syslog/rulebase.rb")
>>>        action(type="omfile" file="/data/syslog/test-syslog.log"
>>> template="testFormat") }
>>> 
>>> rulebase.rb file:
>>> rule=test:%date:word% %host:ipv4% :
>>> %%ASA-%number0:number%-%number1:number%: Denied ICMP 
>>> type=%number2:number%, code=%number3:number% from %origin:ipv4% on 
>>> interface outside
>>> 
>>> test-syslog.log file:
>>> { "origin": "77.2.2.2", "number3": "0", "number2": "8", "number1":
>>> "313001", "number0": "3", "host": "0.0.0.0", "date":
>>> "2013-06-26T10:47:42+01:00" }
>>> 
>>> Message:
>>> "2013-06-26T10:47:42+01:00 0.0.0.0 : %ASA-3-313001: Denied ICMP 
>>> type=8,
>>> code=0 from 77.2.2.2 on interface outside"
>>> ====================================================================
>>> ==
>>> 
>>> 
>>> Thanks in advance for any help,
>>> 
>>> Jose Castillo
>>> 
>>> 
>>> 
>>> 
>> 
>