[Lognorm] Log normalization and the leading space

[Rainer Gerhards]

I am CC'ing the rsyslog mailing list as the issue is more related the
rsyslog and syslog in general. I suggest to subscribe in order to receive
follow-ups.

I think the problem you see is based on the fact that RFC3164 - which is
used to parse these types of messages - specifies that everything after the
TAG is the message. Usually, messages have "TAG: mm", note the space before
mm. This is where it stems from.

In regard to lognorm rules, you can simply duplicate the entries with and
without a space in front. It's a bit ugly, but a work-around you can use
right now.

HTH
Rainer


On Fri, Apr 4, 2014 at 2:36 PM, Davor Saric <davor.saric at srce.hr> wrote:

> Hi,
>
> I have a central rsyslog server, and rsyslog clients that ship their logs
> to central rsyslog. Rsyslog clients on servers are v5 and central rsyslog
> is v7. Central rsyslog sends incoming logs of clients to elasticsearch and
> also ship his own local logs of central server. On clients, I’m using
> imfile modul to read apache logs and also use imfile on central rsyslog
> server to ship his own apache logs to elasticsearch. The problem is that
> apache logs that are coming from clients have a space in msg part so
> normalize rule for those logs is:
> rule=: %client_ip:word% %rlogname:word% %ruser:word% [%apache_date:word%
> %tz:char-to:]%] "%method:word% %url:word% %pver:char-to:"%" %status:word%
> %bytesend:word% "%referrer:char-to:"%" "%useragent:char-to:"%"
>
> And normalize rule for central his own local apache logs is:
> rule=:%client_ip:word% %rlogname:word% %ruser:word% [%apache_date:word%
> %tz:char-to:]%] "%method:word% %url:word% %pver:char-to:"%" %status:word%
> %bytesend:word% "%referrer:char-to:"%" "%useragent:char-to:"%"
>
> The only difference between the rules is that the one that normalize
> incoming apache logs from the clients has one space at first, and the one
> that normalize local apache logs of central rsyslog server has no space.
>
> Here is template for incoming apache logs and the template for local
> apache logs. I had to use position.from=2 because of the space in msg of
> incoming logs. If I use the same template for local apache logs, the first
> character is cut of which is first number of ip adress of client:
>
> template(name="httpd-access_remote" type="list") {
> property(name="msg" position.from="2″)
> constant(value="\n")
> }
>
> template(name="httpd-access_local" type="list") {
> property(name="msg")
> constant(value="\n")
> }
>
> As I can see, the msg property of incoming apache logs have a space at
> beggining but when reading local logs through imfile the msg property
> doesn't have empty space in the beginning.
>
>
> With regards,
> --
> Davor Saric, System Engineer
> Computer Systems Department
>
> SRCE - University of Zagreb University Computing Center, www.srce.unizg.hr
> davor.saric at srce.hr, tel: +385 1 616 58 01, fax: +385 1 616 55 59
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20140404/b5fb3b7e/attachment.html>