[rsyslog] filter logger tags from syslog

[david at lang.hm]

On Wed, 25 Nov 2009, Jose Sanchez wrote:

> Hello,
>
> I've been using classic syslog for centralizing apache access logs from one server to a remote syslog server, the thing is syslog adds some nasty tags before the lines in the access logs and I cant get them off, ie:
>
> "Nov 25 21:25:37 server1 logger:"
>
> I would like to know if rsyslog has the option to filter this kind of stuff, I just want to have the logs sent to the syslog server exactly like I was saving them in a local access.log file.
>
> Thanks in advance.

'logger:' is added by the logger program that apache is using to send the 
logs to syslog.

a properly formatted syslog message will include a timestamp and what 
server it came from (note that the apache logs do _not_ tell you what 
virtual server the log comes from, it usually uses a different file for 
each log, so when you mix them into syslog you won't be able to tell them 
apart)

so you have three basic options

1. let logger do it's default thing and then use a formatting command to 
strip off the 'syslogie' parts to get back to the apache default in the 
file

2. leave the 'syslogie' parts in when you write it to a file and have your 
analysis tool strip them out

3. reformat the apache log message so that you put useful information in 
the 'syslogie' parts of the message.

you can move the timestamp to the beginning (you can do this with or 
without the timezone, the format obviously differs slightly)

you can put the name of the virtual host in the server field

you can replace 'logger:' with something like apache[80]: or apache[443]:

I am going to be setting up something along the lines of #3 in the next 
few weeks. I figure I will also want to tinker with other things in the 
log message. there are items that apache can log, but does not log by 
default (I believe that how long it took to process the request is one of 
these), and also since syslog defaults to limiting log messages to 1-2K 
(depending on your impementation), there are some fields that I would want 
to move late in the message so that if they get very long they don't cause 
other fields to be lost due to truncation (URL and referrer fields can be 
several K long by themselves)

David Lang