[Lognorm] Log normalization and the leading space

[Davor Saric]

Hi,

I have a central rsyslog server, and rsyslog clients that ship their 
logs to central rsyslog. Rsyslog clients on servers are v5 and central 
rsyslog is v7. Central rsyslog sends incoming logs of clients to 
elasticsearch and also ship his own local logs of central server. On 
clients, I’m using imfile modul to read apache logs and also use imfile 
on central rsyslog server to ship his own apache logs to elasticsearch. 
The problem is that apache logs that are coming from clients have a 
space in msg part so normalize rule for those logs is:
rule=: %client_ip:word% %rlogname:word% %ruser:word% [%apache_date:word% 
%tz:char-to:]%] "%method:word% %url:word% %pver:char-to:"%" 
%status:word% %bytesend:word% "%referrer:char-to:"%" "%useragent:char-to:"%"

And normalize rule for central his own local apache logs is:
rule=:%client_ip:word% %rlogname:word% %ruser:word% [%apache_date:word% 
%tz:char-to:]%] "%method:word% %url:word% %pver:char-to:"%" 
%status:word% %bytesend:word% "%referrer:char-to:"%" "%useragent:char-to:"%"

The only difference between the rules is that the one that normalize 
incoming apache logs from the clients has one space at first, and the 
one that normalize local apache logs of central rsyslog server has no space.

Here is template for incoming apache logs and the template for local 
apache logs. I had to use position.from=2 because of the space in msg of 
incoming logs. If I use the same template for local apache logs, the 
first character is cut of which is first number of ip adress of client:

template(name="httpd-access_remote" type="list") {
property(name="msg" position.from="2″)
constant(value="\n")
}

template(name="httpd-access_local" type="list") {
property(name="msg")
constant(value="\n")
}

As I can see, the msg property of incoming apache logs have a space at 
beggining but when reading local logs through imfile the msg property 
doesn't have empty space in the beginning.

With regards,
-- 
Davor Saric, System Engineer
Computer Systems Department

SRCE - University of Zagreb University Computing Center, www.srce.unizg.hr
davor.saric at srce.hr, tel: +385 1 616 58 01, fax: +385 1 616 55 59