[rsyslog] converted %rawsmg% to %msg%

[david at lang.hm]

On Wed, 22 Dec 2010, Rob Shepherd wrote:

> Dear rsyslogd users,
>
> I have a semicolon seperated msg but the first field in the submitted
> string has a space in it.
>
> Here is the %rawmsg%
>
> "<158>Dec 21 22:35:21 x-wing.datacymru.net 2010-12-21
> 22:35:21;2010-12-21
> 22:35:21+0000;null;null;35667;127.0.0.1;USER_ACCESS;USER_LOGGED_IN;User:
> abc at example.com Logged in"
>
> And here is the resulting %msg%
>
> " 22:35:21;2010-12-21
> 22:35:21+0000;null;null;35667;127.0.0.1;USER_ACCESS;USER_LOGGED_IN;User:
> channeladmin0 at zetaplc.com Logged in"
>
> (The double quotes aren't part of the resulting text, I've just put it
> there to indicate the leading <space> in the %msg%)
>
> As you can see the first field has been stripped of it's 'yyyy-MM-dd'
> component.
>
> This makes the DB insert fail
>> pgsql, DBError(silent): db error (0): ERROR:  invalid input syntax for type 
>> timestamp: " 22:49:38" at character 136
>
>
> I've found that the missing text makes it's way to the %app-name% and
> %syslogtag% properties
>
>
> I have a workaround, by using %syslogtag%%msg:F,59:1% to provide the two
> parts as required.
>
> I'd like to not use a workaround if possible.
>
> is there a command the defines how the message is pulled out from the 
> raw.? can I encapsulate my semicolon-delimited string any better to 
> avoid this behaviour?

the problem you are running into is that the syslog RFC defines a properly 
formatted message as being

<###>MMM DD HH:MM:SS hostname syslogtag message

where message can contain just about anything.

the default parser tries very hard to figure out what is what to handle 
malformed messages, but in your case it has no way to know that the firs 
section of the message you want belongs as part of %msg% and there is no 
syslogtag field.

creating a custom format to combine the fields is the easiest way to deal 
with the problem.

The other option is to create a custom parser that will understand these 
messages and put everything into the %msg% field. This is a very new 
capaibility in rsyslog, and so there aren't many examples of how to do 
this yet.

David Lang