From Christopher.Racky at web.de  Fri May  8 15:28:37 2015
From: Christopher.Racky at web.de (Christopher.Racky at web.de)
Date: Fri, 8 May 2015 15:28:37 +0200
Subject: [Lognorm] CSV
Message-ID: <trinity-470f7c35-9793-470d-a94f-de53a9ed3f9d-1431091717299@3capp-webde-bs04>

Hi,

On my rhel6 system I get a segfault:

[root at bug log]# lognormalizer -e csv  -r /etc/rsyslog.rb
May  8 15:11:38 fritz user1: hallo
Segmentation fault

Meanwhile this works as expected:

[root at bug log]# lognormalizer  -r /etc/rsyslog.rb
May  8 15:11:38 fritz user1: hallo
[cee at 115 user="user1:" rcvdfrom="fritz" rcvdat="May  8 15:11:38"]



Seems to be a bug in lognormalizer...

Config file looks like:
prefix=%rcvdat:date-rfc3164% %rcvdfrom:word%
rule=: %user:word% hallo


[root at bug log]# yum info liblognorm1
Loaded plugins: fastestmirror, presto
Loading mirror speeds from cached hostfile
 * base: centos.mirror.linuxwerk.com
 * extras: mirror.de.leaseweb.net
 * updates: ftp.fau.de
Installed Packages
Name        : liblognorm1
Arch        : x86_64
Version     : 1.1.1
Release     : 1.el6
Size        : 88 k
Repo        : installed
>From repo   : rsyslog-v8-stable
Summary     : Fast samples-based log normalization library
URL         : http://www.liblognorm.com
License     : LGPLv2+
Description : Briefly described, liblognorm is a tool to normalize log data.


regards
Chris

From david at lang.hm  Fri May  8 22:19:45 2015
From: david at lang.hm (David Lang)
Date: Fri, 8 May 2015 13:19:45 -0700 (PDT)
Subject: [Lognorm] CSV
In-Reply-To: <trinity-470f7c35-9793-470d-a94f-de53a9ed3f9d-1431091717299@3capp-webde-bs04>
References: <trinity-470f7c35-9793-470d-a94f-de53a9ed3f9d-1431091717299@3capp-webde-bs04>
Message-ID: <alpine.DEB.2.02.1505081317560.20918@nftneq.ynat.uz>

I agree there is a bug in the liblognorm csv output. If I do the same test 
without csv, or using json it works. I don't get a segmentation fault, but I do 
get blank output.

root at ISEC25:~# echo "May  8 15:11:38 fritz user1: hallo" |/usr/lib/lognorm/lognormalizer -e csv  -r del3

root at ISEC25:~# echo "May  8 15:11:38 fritz user1: hallo" |/usr/lib/lognorm/lognormalizer -e json  -r del3
{ "user": "user1:", "rcvdfrom": "fritz", "rcvdat": "May  8 15:11:38" }

David Lang

On Fri, 8 
May 2015, 
Christopher.Racky at web.de wrote:

> Date: Fri, 8 May 2015 15:28:37 +0200
> From: Christopher.Racky at web.de
> Reply-To: lognorm <lognorm at lists.adiscon.com>
> To: lognorm at lists.adiscon.com
> Subject: [Lognorm] CSV
> 
> Hi,
>
> On my rhel6 system I get a segfault:
>
> [root at bug log]# lognormalizer -e csv  -r /etc/rsyslog.rb
> May  8 15:11:38 fritz user1: hallo
> Segmentation fault
>
> Meanwhile this works as expected:
>
> [root at bug log]# lognormalizer  -r /etc/rsyslog.rb
> May  8 15:11:38 fritz user1: hallo
> [cee at 115 user="user1:" rcvdfrom="fritz" rcvdat="May  8 15:11:38"]
>
>
>
> Seems to be a bug in lognormalizer...
>
> Config file looks like:
> prefix=%rcvdat:date-rfc3164% %rcvdfrom:word%
> rule=: %user:word% hallo
>
>
> [root at bug log]# yum info liblognorm1
> Loaded plugins: fastestmirror, presto
> Loading mirror speeds from cached hostfile
> * base: centos.mirror.linuxwerk.com
> * extras: mirror.de.leaseweb.net
> * updates: ftp.fau.de
> Installed Packages
> Name        : liblognorm1
> Arch        : x86_64
> Version     : 1.1.1
> Release     : 1.el6
> Size        : 88 k
> Repo        : installed
> From repo   : rsyslog-v8-stable
> Summary     : Fast samples-based log normalization library
> URL         : http://www.liblognorm.com
> License     : LGPLv2+
> Description : Briefly described, liblognorm is a tool to normalize log data.
>
>
> regards
> Chris
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm
>

From pavel at levshin.spb.ru  Sat May  9 09:58:40 2015
From: pavel at levshin.spb.ru (Pavel Levshin)
Date: Sat, 09 May 2015 10:58:40 +0300
Subject: [Lognorm] CSV
In-Reply-To: <alpine.DEB.2.02.1505081317560.20918@nftneq.ynat.uz>
References: <trinity-470f7c35-9793-470d-a94f-de53a9ed3f9d-1431091717299@3capp-webde-bs04>
	<alpine.DEB.2.02.1505081317560.20918@nftneq.ynat.uz>
Message-ID: <554DBE30.1080303@levshin.spb.ru>


For CSV, there should be field list in command options (-E). CSV output 
does not make sense when it does not know what to output.

It must not segfault, though.


--
Pavel


08.05.2015 23:19, David Lang:
> I agree there is a bug in the liblognorm csv output. If I do the same 
> test without csv, or using json it works. I don't get a segmentation 
> fault, but I do get blank output.
>
> root at ISEC25:~# echo "May  8 15:11:38 fritz user1: hallo" 
> |/usr/lib/lognorm/lognormalizer -e csv  -r del3
>
> root at ISEC25:~# echo "May  8 15:11:38 fritz user1: hallo" 
> |/usr/lib/lognorm/lognormalizer -e json  -r del3
> { "user": "user1:", "rcvdfrom": "fritz", "rcvdat": "May  8 15:11:38" }
>
> David Lang
>
> On Fri, 8 May 2015, Christopher.Racky at web.de wrote:
>
>> Date: Fri, 8 May 2015 15:28:37 +0200
>> From: Christopher.Racky at web.de
>> Reply-To: lognorm <lognorm at lists.adiscon.com>
>> To: lognorm at lists.adiscon.com
>> Subject: [Lognorm] CSV
>>
>> Hi,
>>
>> On my rhel6 system I get a segfault:
>>
>> [root at bug log]# lognormalizer -e csv  -r /etc/rsyslog.rb
>> May  8 15:11:38 fritz user1: hallo
>> Segmentation fault
>>
>> Meanwhile this works as expected:
>>
>> [root at bug log]# lognormalizer  -r /etc/rsyslog.rb
>> May  8 15:11:38 fritz user1: hallo
>> [cee at 115 user="user1:" rcvdfrom="fritz" rcvdat="May  8 15:11:38"]
>>
>>
>>
>> Seems to be a bug in lognormalizer...
>>
>> Config file looks like:
>> prefix=%rcvdat:date-rfc3164% %rcvdfrom:word%
>> rule=: %user:word% hallo
>>
>>
>> [root at bug log]# yum info liblognorm1
>> Loaded plugins: fastestmirror, presto
>> Loading mirror speeds from cached hostfile
>> * base: centos.mirror.linuxwerk.com
>> * extras: mirror.de.leaseweb.net
>> * updates: ftp.fau.de
>> Installed Packages
>> Name        : liblognorm1
>> Arch        : x86_64
>> Version     : 1.1.1
>> Release     : 1.el6
>> Size        : 88 k
>> Repo        : installed
>> From repo   : rsyslog-v8-stable
>> Summary     : Fast samples-based log normalization library
>> URL         : http://www.liblognorm.com
>> License     : LGPLv2+
>> Description : Briefly described, liblognorm is a tool to normalize 
>> log data.
>>
>>
>> regards
>> Chris
>> _______________________________________________
>> Lognorm mailing list
>> Lognorm at lists.adiscon.com
>> http://lists.adiscon.net/mailman/listinfo/lognorm
>>
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm

From Christopher.Racky at web.de  Mon May 11 11:59:23 2015
From: Christopher.Racky at web.de (Christopher.Racky at web.de)
Date: Mon, 11 May 2015 11:59:23 +0200
Subject: [Lognorm] lognormalize and unrelevant rest of a message...
Message-ID: <trinity-90f5d526-497d-4003-9a5d-bbddefca19ce-1431338363383@3capp-webde-bs48>

Hi,

I have several times the situation, that the message I want to normalize has got an "unrelevant" suffix, that I don't need.

My first expectation was, that lognormalize matches for messages if all the prefix is the same. This is not the case.
After that I tried it with these rules, but none of them seems to work.

[root at bug ~]# cat /etc/rsyslog3.rb
prefix=<%prio:number%>%rcvdat:date-rfc3164% %rcvdfrom:word%
rule=: %user:char-to::%: hallo1 %dontcare:char-to:\xa%
rule=: %user:char-to::%: hallo2 %dontcare:char-to:\xd%
rule=: %user:char-to::%: hallo3 %dontcare:rest%

As you can see, it does not work :-(
[root at bug ~]# echo "<38>May  8 14:32:38 fritz username: hallo1 asd" |  lognormalizer -r /etc/rsyslog3.rb
[cee at 115 originalmsg="<38>May  8 14:32:38 fritz username: hallo1 asd" unparsed-data="asd"]
[root at bug ~]# echo "<38>May  8 14:32:38 fritz username: hallo2 asd" |  lognormalizer -r /etc/rsyslog3.rb
[cee at 115 originalmsg="<38>May  8 14:32:38 fritz username: hallo2 asd" unparsed-data="asd"]
[root at bug ~]# echo "<38>May  8 14:32:38 fritz username: hallo3 asd" |  lognormalizer -r /etc/rsyslog3.rb
[cee at 115 originalmsg="<38>May  8 14:32:38 fritz username: hallo3 asd" unparsed-data="asd"]


The documentation does not really mention something about that. From the comments about "rest" i think it should work, because it more tends to "match" to realy in liblognormalize 1.1...
Do you have any hint how it is possible to "ignore" the rest of the message?

Best regards
Chris

From rgerhards at hq.adiscon.com  Mon May 11 12:01:31 2015
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Mon, 11 May 2015 12:01:31 +0200
Subject: [Lognorm] lognormalize and unrelevant rest of a message...
In-Reply-To: <trinity-90f5d526-497d-4003-9a5d-bbddefca19ce-1431338363383@3capp-webde-bs48>
References: <trinity-90f5d526-497d-4003-9a5d-bbddefca19ce-1431338363383@3capp-webde-bs48>
Message-ID: <CADk+mPAXrBYrVkGj_LHtqrfBaXM8Ve1mecO8rKRTOfE7_g7UUQ@mail.gmail.com>

"rest" was more or less random in all released versions (besides that
I always caution against using "rest", IMHO it's way too broad.
Anyhow, git master contains an improved "rest" matcher. I suggest you
build from git source and see if that solves your problem.

HTH
Rainer

2015-05-11 11:59 GMT+02:00  <Christopher.Racky at web.de>:
> Hi,
>
> I have several times the situation, that the message I want to normalize has got an "unrelevant" suffix, that I don't need.
>
> My first expectation was, that lognormalize matches for messages if all the prefix is the same. This is not the case.
> After that I tried it with these rules, but none of them seems to work.
>
> [root at bug ~]# cat /etc/rsyslog3.rb
> prefix=<%prio:number%>%rcvdat:date-rfc3164% %rcvdfrom:word%
> rule=: %user:char-to::%: hallo1 %dontcare:char-to:\xa%
> rule=: %user:char-to::%: hallo2 %dontcare:char-to:\xd%
> rule=: %user:char-to::%: hallo3 %dontcare:rest%
>
> As you can see, it does not work :-(
> [root at bug ~]# echo "<38>May  8 14:32:38 fritz username: hallo1 asd" |  lognormalizer -r /etc/rsyslog3.rb
> [cee at 115 originalmsg="<38>May  8 14:32:38 fritz username: hallo1 asd" unparsed-data="asd"]
> [root at bug ~]# echo "<38>May  8 14:32:38 fritz username: hallo2 asd" |  lognormalizer -r /etc/rsyslog3.rb
> [cee at 115 originalmsg="<38>May  8 14:32:38 fritz username: hallo2 asd" unparsed-data="asd"]
> [root at bug ~]# echo "<38>May  8 14:32:38 fritz username: hallo3 asd" |  lognormalizer -r /etc/rsyslog3.rb
> [cee at 115 originalmsg="<38>May  8 14:32:38 fritz username: hallo3 asd" unparsed-data="asd"]
>
>
> The documentation does not really mention something about that. From the comments about "rest" i think it should work, because it more tends to "match" to realy in liblognormalize 1.1...
> Do you have any hint how it is possible to "ignore" the rest of the message?
>
> Best regards
> Chris
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm

From pavel at levshin.spb.ru  Mon May 11 12:36:39 2015
From: pavel at levshin.spb.ru (Pavel Levshin)
Date: Mon, 11 May 2015 13:36:39 +0300
Subject: [Lognorm] lognormalize and unrelevant rest of a message...
In-Reply-To: <trinity-90f5d526-497d-4003-9a5d-bbddefca19ce-1431338363383@3capp-webde-bs48>
References: <trinity-90f5d526-497d-4003-9a5d-bbddefca19ce-1431338363383@3capp-webde-bs48>
Message-ID: <55508637.3050302@levshin.spb.ru>


It had been working once in 1.0, I believe.

lpk at freak:~$ echo "<38>May  8 14:32:38 fritz username: hallo1 asd" |  
lognormalizer -r rsyslog3.rb
[cee at 115 originalmsg="<38>May  8 14:32:38 fritz username: hallo1 asd" 
unparsed-data="asd"]
lpk at freak:~$ echo "<38>May  8 14:32:38 fritz username: hallo2 asd" |  
lognormalizer -r rsyslog3.rb
[cee at 115 originalmsg="<38>May  8 14:32:38 fritz username: hallo2 asd" 
unparsed-data="asd"]
lpk at freak:~$ echo "<38>May  8 14:32:38 fritz username: hallo3 asd" |  
lognormalizer -r rsyslog3.rb
[cee at 115 dontcare="asd" user="username" rcvdfrom="fritz" rcvdat="May  8 
14:32:38" prio="38"]

It was supposed to work this way, because there was no other means to 
parse partially structured strings.

Also, if you are trying to debug this, verbose output (-v) can be useful.

--
Pavel


11.05.2015 12:59, Christopher.Racky at web.de:
> Hi,
>
> I have several times the situation, that the message I want to normalize has got an "unrelevant" suffix, that I don't need.
>
> My first expectation was, that lognormalize matches for messages if all the prefix is the same. This is not the case.
> After that I tried it with these rules, but none of them seems to work.
>
> [root at bug ~]# cat /etc/rsyslog3.rb
> prefix=<%prio:number%>%rcvdat:date-rfc3164% %rcvdfrom:word%
> rule=: %user:char-to::%: hallo1 %dontcare:char-to:\xa%
> rule=: %user:char-to::%: hallo2 %dontcare:char-to:\xd%
> rule=: %user:char-to::%: hallo3 %dontcare:rest%
>
> As you can see, it does not work :-(
> [root at bug ~]# echo "<38>May  8 14:32:38 fritz username: hallo1 asd" |  lognormalizer -r /etc/rsyslog3.rb
> [cee at 115 originalmsg="<38>May  8 14:32:38 fritz username: hallo1 asd" unparsed-data="asd"]
> [root at bug ~]# echo "<38>May  8 14:32:38 fritz username: hallo2 asd" |  lognormalizer -r /etc/rsyslog3.rb
> [cee at 115 originalmsg="<38>May  8 14:32:38 fritz username: hallo2 asd" unparsed-data="asd"]
> [root at bug ~]# echo "<38>May  8 14:32:38 fritz username: hallo3 asd" |  lognormalizer -r /etc/rsyslog3.rb
> [cee at 115 originalmsg="<38>May  8 14:32:38 fritz username: hallo3 asd" unparsed-data="asd"]
>
>
> The documentation does not really mention something about that. From the comments about "rest" i think it should work, because it more tends to "match" to realy in liblognormalize 1.1...
> Do you have any hint how it is possible to "ignore" the rest of the message?
>
> Best regards
> Chris
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm

From david at lang.hm  Tue May 12 03:27:05 2015
From: david at lang.hm (David Lang)
Date: Mon, 11 May 2015 18:27:05 -0700 (PDT)
Subject: [Lognorm] lognormalize and unrelevant rest of a message...
In-Reply-To: <trinity-90f5d526-497d-4003-9a5d-bbddefca19ce-1431338363383@3capp-webde-bs48>
References: <trinity-90f5d526-497d-4003-9a5d-bbddefca19ce-1431338363383@3capp-webde-bs48>
Message-ID: <alpine.DEB.2.02.1505111825360.6441@nftneq.ynat.uz>

On Mon, 11 May 2015, Christopher.Racky at web.de wrote:

> Hi,
>
> I have several times the situation, that the message I want to normalize has got an "unrelevant" suffix, that I don't need.
>
> My first expectation was, that lognormalize matches for messages if all the prefix is the same. This is not the case.
> After that I tried it with these rules, but none of them seems to work.
>
> [root at bug ~]# cat /etc/rsyslog3.rb
> prefix=<%prio:number%>%rcvdat:date-rfc3164% %rcvdfrom:word%
> rule=: %user:char-to::%: hallo1 %dontcare:char-to:\xa%
> rule=: %user:char-to::%: hallo2 %dontcare:char-to:\xd%
> rule=: %user:char-to::%: hallo3 %dontcare:rest%
>
> As you can see, it does not work :-(
> [root at bug ~]# echo "<38>May  8 14:32:38 fritz username: hallo1 asd" |  lognormalizer -r /etc/rsyslog3.rb
> [cee at 115 originalmsg="<38>May  8 14:32:38 fritz username: hallo1 asd" unparsed-data="asd"]
> [root at bug ~]# echo "<38>May  8 14:32:38 fritz username: hallo2 asd" |  lognormalizer -r /etc/rsyslog3.rb
> [cee at 115 originalmsg="<38>May  8 14:32:38 fritz username: hallo2 asd" unparsed-data="asd"]
> [root at bug ~]# echo "<38>May  8 14:32:38 fritz username: hallo3 asd" |  lognormalizer -r /etc/rsyslog3.rb
> [cee at 115 originalmsg="<38>May  8 14:32:38 fritz username: hallo3 asd" unparsed-data="asd"]
>
>
> The documentation does not really mention something about that. From the comments about "rest" i think it should work, because it more tends to "match" to realy in liblognormalize 1.1...
> Do you have any hint how it is possible to "ignore" the rest of the message?

compiled from git last week

# echo "<38>May  8 14:32:38 fritz username: hallo3 asd" | 
/usr/lib/lognorm/lognormalizer -r del4
[cee at 115 dontcare="asd" user="username" rcvdfrom="fritz" rcvdat="May  8 
14:32:38" prio="38"]

also, if you change 'dontcare' to '-' it throws the value away

# echo "<38>May  8 14:32:38 fritz username: hallo3 asd" | 
/usr/lib/lognorm/lognormalizer -r del4
[cee at 115 user="username" rcvdfrom="fritz" rcvdat="May  8 14:32:38" prio="38"]

# cat del4
prefix=<%prio:number%>%rcvdat:date-rfc3164% %rcvdfrom:word%
rule=: %user:char-to::%: hallo1 %dontcare:char-to:\xa%
rule=: %user:char-to::%: hallo2 %dontcare:char-to:\xd%
rule=: %user:char-to::%: hallo3 %-:rest%

David Lang

From Christopher.Racky at web.de  Wed May 13 09:34:18 2015
From: Christopher.Racky at web.de (Christopher.Racky at web.de)
Date: Wed, 13 May 2015 09:34:18 +0200
Subject: [Lognorm] Bugs in lognormalize and/or rsyslog bracket / Whitespace
Message-ID: <trinity-5904690e-5cb3-4f03-9b3f-69f0cbee1007-1431502458319@3capp-webde-bs27>

Hi,
I'm right now defining some normalization rules.
Actually I'm still using the latest released version for red-hat, because on my system it seems to be quite complex to compile a binary version... Daily snapshot RPMs for RHEL would be great ;)


Some strange things happen:

1. *** Issue with bracket ***:
[root at bug ~]# echo "<86>Apr 29 09:17:14 hostname atd[32016]: pam_unix(atd:session): session opened for user root by (uid=0)" | lognormalizer -r /etc/rsyslog4.rb
[cee at 115 event.tags="atd,sesopen" uid="0" tuser="root" processid="32016" rcvdfrom="hostname" rcvdat="Apr 29 09:17:14" prio="86" process="atd" action="session opened"]

==> Works great
Meanwhile the same rule (nothing changed in file) does not work with rsyslog8.9.
Issue is in the suffix: (uid=0).

Rule is:
rule=: atd[%processid:number%]: pam_unix(atd:session): session opened for user %tuser:word% by (uid=%uid:number%)

This rule, works for both (rsyslog and lognormalize):
rule=: atd[%processid:number%]: pam_unix(atd:session): session opened for user %tuser:word% by %uid:word%


2. *** Issues with whitespace ***

Data:
 sudo:   username

Rules:
rule=: sudo:%-:whitespace%%user:word%
rule=: sudo: %-:whitespace% %user:word%


[root at bug ~]# echo "<85>May 12 13:23:06 fritz sudo:  username" |lognormalizer -r /etc/rsyslog4.rb
[cee at 115 originalmsg="<85>May 12 13:23:06 fritz sudo:  username" unparsed-data=""]

3. "rest" topic is already known... 


best regards
Chris

From rgerhards at hq.adiscon.com  Wed May 13 11:29:54 2015
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Wed, 13 May 2015 11:29:54 +0200
Subject: [Lognorm] Bugs in lognormalize and/or rsyslog bracket /
	Whitespace
In-Reply-To: <trinity-5904690e-5cb3-4f03-9b3f-69f0cbee1007-1431502458319@3capp-webde-bs27>
References: <trinity-5904690e-5cb3-4f03-9b3f-69f0cbee1007-1431502458319@3capp-webde-bs27>
Message-ID: <CADk+mPBzFLJf9fCZ1_A=8DUUv1Ujs=_uf1BNx=4oVETgLd2SjA@mail.gmail.com>

2015-05-13 9:34 GMT+02:00  <Christopher.Racky at web.de>:
> Hi,
> I'm right now defining some normalization rules.
> Actually I'm still using the latest released version for red-hat, because on my system it seems to be quite complex to compile a binary version... Daily snapshot RPMs for RHEL would be great ;)
>
>
> Some strange things happen:
>
> 1. *** Issue with bracket ***:
> [root at bug ~]# echo "<86>Apr 29 09:17:14 hostname atd[32016]: pam_unix(atd:session): session opened for user root by (uid=0)" | lognormalizer -r /etc/rsyslog4.rb
> [cee at 115 event.tags="atd,sesopen" uid="0" tuser="root" processid="32016" rcvdfrom="hostname" rcvdat="Apr 29 09:17:14" prio="86" process="atd" action="session opened"]
>
> ==> Works great
> Meanwhile the same rule (nothing changed in file) does not work with rsyslog8.9.
> Issue is in the suffix: (uid=0).

I don't understand what you mean here (in regard to suffix).

Is this with the same liblognorm? Than it's really strange. I suggest
to open a github issue tracker

if it's with the same version of liblognorm --> issue for rsyslog
otherwise --> issue for liblognorm

but please explain what I did not understand.

>
> Rule is:
> rule=: atd[%processid:number%]: pam_unix(atd:session): session opened for user %tuser:word% by (uid=%uid:number%)
>
> This rule, works for both (rsyslog and lognormalize):
> rule=: atd[%processid:number%]: pam_unix(atd:session): session opened for user %tuser:word% by %uid:word%
>
>
> 2. *** Issues with whitespace ***
>
> Data:
>  sudo:   username
>
> Rules:
> rule=: sudo:%-:whitespace%%user:word%
> rule=: sudo: %-:whitespace% %user:word%
>
>
> [root at bug ~]# echo "<85>May 12 13:23:06 fritz sudo:  username" |lognormalizer -r /etc/rsyslog4.rb
> [cee at 115 originalmsg="<85>May 12 13:23:06 fritz sudo:  username" unparsed-data=""]

I guess that's another incartnation of
https://github.com/rsyslog/liblognorm/issues/57

>
> 3. "rest" topic is already known...
and fixed (at least as far as possible for such a broad syntax)! Patch
is in github.

Rainer
>
>
> best regards
> Chris
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm

From Christopher.Racky at web.de  Fri May 15 07:35:04 2015
From: Christopher.Racky at web.de (Christopher.Racky at web.de)
Date: Fri, 15 May 2015 07:35:04 +0200
Subject: [Lognorm] Bugs in lognormalize and/or rsyslog bracket /
 Whitespace
In-Reply-To: <CADk+mPBzFLJf9fCZ1_A=8DUUv1Ujs=_uf1BNx=4oVETgLd2SjA@mail.gmail.com>
References: <trinity-5904690e-5cb3-4f03-9b3f-69f0cbee1007-1431502458319@3capp-webde-bs27>,
	<CADk+mPBzFLJf9fCZ1_A=8DUUv1Ujs=_uf1BNx=4oVETgLd2SjA@mail.gmail.com>
Message-ID: <trinity-e7465506-276f-4537-99d7-59dacd51eeaa-1431668104717@3capp-webde-bs26>


> Gesendet: Mittwoch, 13. Mai 2015 um 11:29 Uhr
> Von: "Rainer Gerhards" <rgerhards at hq.adiscon.com>
> An: lognorm <lognorm at lists.adiscon.com>
> Betreff: Re: [Lognorm] Bugs in lognormalize and/or rsyslog bracket / Whitespace
> > Hi,
> > I'm right now defining some normalization rules.
> > Actually I'm still using the latest released version for red-hat, because on my system it seems to be quite complex to compile a binary version... Daily snapshot RPMs for RHEL would be great ;)
> > 1. *** Issue with bracket ***:
> > [root at bug ~]# echo "<86>Apr 29 09:17:14 hostname atd[32016]: pam_unix(atd:session): session opened for user root by (uid=0)" | lognormalizer -r /etc/rsyslog4.rb
> > [cee at 115 event.tags="atd,sesopen" uid="0" tuser="root" processid="32016" rcvdfrom="hostname" rcvdat="Apr 29 09:17:14" prio="86" process="atd" action="session opened"]
> >
> > ==> Works great
> > Meanwhile the same rule (nothing changed in file) does not work with rsyslog8.9.
> > Issue is in the suffix: (uid=0).
> 
> I don't understand what you mean here (in regard to suffix).
> 
> Is this with the same liblognorm? Than it's really strange. I suggest
> to open a github issue tracker
> 
> if it's with the same version of liblognorm --> issue for rsyslog
> otherwise --> issue for liblognorm
> 
> but please explain what I did not understand.

Yes, the rule works fine with lognormalizer command, but not in rsyslog.
I will open a Ticket for that at rsylog. All other rules (about 30 rules) works fine.
If I reduce the message from
"<86>Apr 29 09:17:14 hostname atd[32016]: pam_unix(atd:session): session opened for user root by (uid=0)"
to
"<86>Apr 29 09:17:14 hostname atd[32016]: pam_unix(atd:session): session opened for user root by "
by using the reduced filter:
 prefix:=...
 rule=: atd[%processid:number%]: pam_unix(atd:session): session opened for user %tuser:word% by 
matching in rsyslog (and lognormalizer) works.
So with suffix, I did not mean the keyword in liblognormalizer, just the suffix/ending of the string, in characters: (uid=0)

Chris


> I guess that's another incartnation of
> https://github.com/rsyslog/liblognorm/issues/57
> 
> >
> > 3. "rest" topic is already known...
> and fixed (at least as far as possible for such a broad syntax)! Patch
> is in github.
> 
> Rainer
> >
> >
> > best regards
> > Chris
> > _______________________________________________
> > Lognorm mailing list
> > Lognorm at lists.adiscon.com
> > http://lists.adiscon.net/mailman/listinfo/lognorm
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm
> 

From rgerhards at hq.adiscon.com  Fri May 15 08:46:24 2015
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Fri, 15 May 2015 08:46:24 +0200
Subject: [Lognorm] Bugs in lognormalize and/or rsyslog bracket /
	Whitespace
In-Reply-To: <trinity-e7465506-276f-4537-99d7-59dacd51eeaa-1431668104717@3capp-webde-bs26>
References: <trinity-5904690e-5cb3-4f03-9b3f-69f0cbee1007-1431502458319@3capp-webde-bs27>
	<CADk+mPBzFLJf9fCZ1_A=8DUUv1Ujs=_uf1BNx=4oVETgLd2SjA@mail.gmail.com>
	<trinity-e7465506-276f-4537-99d7-59dacd51eeaa-1431668104717@3capp-webde-bs26>
Message-ID: <CADk+mPD5WcJgVYv8Bs8GQWyL=7SLHqhWeHwEK7qA1XJBfY8few@mail.gmail.com>

2015-05-15 7:35 GMT+02:00  <Christopher.Racky at web.de>:
>
>> Gesendet: Mittwoch, 13. Mai 2015 um 11:29 Uhr
>> Von: "Rainer Gerhards" <rgerhards at hq.adiscon.com>
>> An: lognorm <lognorm at lists.adiscon.com>
>> Betreff: Re: [Lognorm] Bugs in lognormalize and/or rsyslog bracket / Whitespace
>> > Hi,
>> > I'm right now defining some normalization rules.
>> > Actually I'm still using the latest released version for red-hat, because on my system it seems to be quite complex to compile a binary version... Daily snapshot RPMs for RHEL would be great ;)
>> > 1. *** Issue with bracket ***:
>> > [root at bug ~]# echo "<86>Apr 29 09:17:14 hostname atd[32016]: pam_unix(atd:session): session opened for user root by (uid=0)" | lognormalizer -r /etc/rsyslog4.rb
>> > [cee at 115 event.tags="atd,sesopen" uid="0" tuser="root" processid="32016" rcvdfrom="hostname" rcvdat="Apr 29 09:17:14" prio="86" process="atd" action="session opened"]
>> >
>> > ==> Works great
>> > Meanwhile the same rule (nothing changed in file) does not work with rsyslog8.9.
>> > Issue is in the suffix: (uid=0).
>>
>> I don't understand what you mean here (in regard to suffix).
>>
>> Is this with the same liblognorm? Than it's really strange. I suggest
>> to open a github issue tracker
>>
>> if it's with the same version of liblognorm --> issue for rsyslog
>> otherwise --> issue for liblognorm
>>
>> but please explain what I did not understand.
>
> Yes, the rule works fine with lognormalizer command, but not in rsyslog.
> I will open a Ticket for that at rsylog.

I would guess that the problem is rooted in a proptery that is
different than you expect. I suggest to ask that question first on the
rsyslog mailing list, chances are good we can nail it down there ( I
don't want to do it here).

Rainer

> All other rules (about 30 rules) works fine.
> If I reduce the message from
> "<86>Apr 29 09:17:14 hostname atd[32016]: pam_unix(atd:session): session opened for user root by (uid=0)"
> to
> "<86>Apr 29 09:17:14 hostname atd[32016]: pam_unix(atd:session): session opened for user root by "
> by using the reduced filter:
>  prefix:=...
>  rule=: atd[%processid:number%]: pam_unix(atd:session): session opened for user %tuser:word% by
> matching in rsyslog (and lognormalizer) works.
> So with suffix, I did not mean the keyword in liblognormalizer, just the suffix/ending of the string, in characters: (uid=0)
>
> Chris
>
>
>> I guess that's another incartnation of
>> https://github.com/rsyslog/liblognorm/issues/57
>>
>> >
>> > 3. "rest" topic is already known...
>> and fixed (at least as far as possible for such a broad syntax)! Patch
>> is in github.
>>
>> Rainer
>> >
>> >
>> > best regards
>> > Chris
>> > _______________________________________________
>> > Lognorm mailing list
>> > Lognorm at lists.adiscon.com
>> > http://lists.adiscon.net/mailman/listinfo/lognorm
>> _______________________________________________
>> Lognorm mailing list
>> Lognorm at lists.adiscon.com
>> http://lists.adiscon.net/mailman/listinfo/lognorm
>>
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm

From Christopher.Racky at web.de  Fri May 15 13:33:49 2015
From: Christopher.Racky at web.de (Christopher.Racky at web.de)
Date: Fri, 15 May 2015 13:33:49 +0200
Subject: [Lognorm] Bugs in lognormalize and/or rsyslog bracket /
 Whitespace
In-Reply-To: <CADk+mPD5WcJgVYv8Bs8GQWyL=7SLHqhWeHwEK7qA1XJBfY8few@mail.gmail.com>
References: <trinity-5904690e-5cb3-4f03-9b3f-69f0cbee1007-1431502458319@3capp-webde-bs27>
	<CADk+mPBzFLJf9fCZ1_A=8DUUv1Ujs=_uf1BNx=4oVETgLd2SjA@mail.gmail.com>
	<trinity-e7465506-276f-4537-99d7-59dacd51eeaa-1431668104717@3capp-webde-bs26>,
	<CADk+mPD5WcJgVYv8Bs8GQWyL=7SLHqhWeHwEK7qA1XJBfY8few@mail.gmail.com>
Message-ID: <trinity-3cb6d0ff-30ef-43a5-be5d-f18b1e537062-1431689629752@3capp-webde-bs26>


>I would guess that the problem is rooted in a proptery that is
>different than you expect. I suggest to ask that question first on the
>rsyslog mailing list, chances are good we can nail it down there ( I
>don't want to do it here).
>
>Rainer
>

Thanks Rainer.
Message was send.

regards
Chris