From adam at spreedly.com  Mon Apr 11 16:49:25 2016
From: adam at spreedly.com (Adam Williams)
Date: Mon, 11 Apr 2016 10:49:25 -0400
Subject: [Lognorm] Single Quote
Message-ID: <CADXju_wbuF7ett0w2LXB6FDhO7Jb0ByEhWdVTn2Df7bSh7Uxjg@mail.gmail.com>

Hello,

I'm having a very difficult time discerning from the documentation at
http://www.liblognorm.com/files/manual/configuration.html exactly how
to get the results I would expect from a very simple example.

    $ echo " event='nginx request'" | /usr/lib/lognorm/lognormalizer
-r /etc/rsyslog.d/normalize_nginx.rules
    [cee at 115 originalmsg=" event='nginx request'" unparsed-data="request'"]

The rules file is:

    version=2
    rule=: event='nginx %event:word%'

I note in the documentation that there are some rule examples with a '
(single quote) at the end of the line, where the sample data has no
such ending character.

    rule=:%f:ref'
    CEF:0|Vendor|Product|Version|Signature ID|some name|Severity|
aa=field1 bb=this is a value cc=field 3

However, there is no documentation explaining what that means.

I have success when I remove the single quotes from my sample data and rule.

    $ echo " event=nginx request" | /usr/lib/lognorm/lognormalizer -r
/etc/rsyslog.d/normalize_nginx.rules
    [cee at 115 auth="request"]

Please help me to see what I am missing, thank you.

Adam

From adam at spreedly.com  Mon Apr 11 18:19:04 2016
From: adam at spreedly.com (Adam Williams)
Date: Mon, 11 Apr 2016 12:19:04 -0400
Subject: [Lognorm] op-quoted-string with single quotes
Message-ID: <CADXju_wVnezg9CWyACyRu79E03htJeP6fGYxMgVev_+ygT6s0w@mail.gmail.com>

Hello,

I have lines like this:

    app=billing description='something long in here'

I'd like to use something shown in the documentation that doesn't seem
to actually work,
http://www.liblognorm.com/files/manual/configuration.html#string

   rule=app=%app:string{"quoting.char.begin":"'",
"quoting.char.end":"'"}%
description=%desc:string{"quoting.char.begin":"'",
"quoting.char.end":"'"}%

However, I get the following error:

    liblognorm error: invalid field type 'string{"quoting.char.begin"'

Then I find that we're told there is no string type:
https://github.com/rsyslog/liblognorm/issues/169

This led me to attempt:

    op-quoted-string{"quoting.char.begin":"'", "quoting.char.end":"'"}
    liblognorm error: invalid field type 'op-quoted-string{"quoting.char.begin"'

Is it possible to accomplish what I need to accomplish? Thanks!

Adam

From david at lang.hm  Mon Apr 11 22:58:09 2016
From: david at lang.hm (David Lang)
Date: Mon, 11 Apr 2016 13:58:09 -0700 (PDT)
Subject: [Lognorm] op-quoted-string with single quotes
In-Reply-To: <CADXju_wVnezg9CWyACyRu79E03htJeP6fGYxMgVev_+ygT6s0w@mail.gmail.com>
References: <CADXju_wVnezg9CWyACyRu79E03htJeP6fGYxMgVev_+ygT6s0w@mail.gmail.com>
Message-ID: <alpine.DEB.2.02.1604111355540.19654@nftneq.ynat.uz>

On Mon, 11 Apr 2016, Adam Williams wrote:

> Hello,
>
> I have lines like this:
>
>    app=billing description='something long in here'
>
> I'd like to use something shown in the documentation that doesn't seem
> to actually work,
> http://www.liblognorm.com/files/manual/configuration.html#string
>
>   rule=app=%app:string{"quoting.char.begin":"'",
> "quoting.char.end":"'"}%
> description=%desc:string{"quoting.char.begin":"'",
> "quoting.char.end":"'"}%
>
> However, I get the following error:
>
>    liblognorm error: invalid field type 'string{"quoting.char.begin"'
>
> Then I find that we're told there is no string type:
> https://github.com/rsyslog/liblognorm/issues/169
>
> This led me to attempt:
>
>    op-quoted-string{"quoting.char.begin":"'", "quoting.char.end":"'"}
>    liblognorm error: invalid field type 'op-quoted-string{"quoting.char.begin"'
>
> Is it possible to accomplish what I need to accomplish? Thanks!

This syntax only works with the unreleased version 2 of liblognorm.

First make sure that you are running that (It is available in the nightly 
releases), and then make sure you have version=2 as the first line of your 
ruleset.

I've been running it for a few months in production, and it is significantly 
more powerful than version 1. I believe that the plan is to release version 2 in 
the next week or so along with rsyslog 8.18, but it could slip one more release.

David Lang