From Christopher.Racky at web.de  Mon Jun  1 21:21:56 2015
From: Christopher.Racky at web.de (Christopher.Racky at web.de)
Date: Mon, 1 Jun 2015 21:21:56 +0200
Subject: [Lognorm] normalization of vmware date format
Message-ID: <trinity-c2e04f2b-593e-4c22-af8a-9b63ddc9cba5-1433186516024@3capp-webde-bs48>

Hi list,

VMWare ESXi Systems have raw log messages like this:
  <35>2015-05-18T07:56:48Z hostname.fqdn.de DCUI: Authentication of user root failed
The date-format does not match to the "standard" Format of any RFC (as far as I see).

How can I add this date into _one_ field via log-normalization?
A rule like this
 rule=:<%prio:number%>%time:char-to:Z%Z %host:word% ...
is to generic, but this "variation" of RFC5424 is not supported by this kind of rule:
 rule=:<%prio:number%>%time:date-rfc5424 %host:word% ...
as you can see here:
[root at bug log]# echo "<35>2015-05-18T07:56:48Z host DCUI: Authentication of user root failed" | lognormalizer -r /etc/csiem.rb
[cee at 115 originalmsg="<35>2015-05-18T07:56:48Z host DCUI: Authentication of user root failed" unparsed-data="56:48Z host DCUI: Authentication of user root failed"]

Because I later want to insert this date via rsyslog ommysql I guess there is also missing converting support. Right?



By the way:
When will the next version of liblognorm be released? Beginning of May seems to over ;) 
There are a couple of important bugfixes of the new version...

best regards
Chris

From fvl at mail.ru  Wed Jun 10 08:10:27 2015
From: fvl at mail.ru (=?UTF-8?B?0KTQsNC00LXQtdCyINCS0LjRgtCw0LvQuNC5INCb0YzQstC+0LLQuNGH?=)
Date: Wed, 10 Jun 2015 09:10:27 +0300
Subject: [Lognorm] =?utf-8?q?Escape_=5B_and_=5D?=
Message-ID: <1433916627.609174282@f94.i.mail.ru>


Hi!
I want to log all from apache. I use custom log in apache that looks like:

[2015-06-09 18:27:07 197 NOVT] [192.168.1.67] [192.168.1.67] [192.168.1.254] [818] [/var/www/host/css/button.css] [192.168.1.67] [HTTP/1.1] [1] [-] [GET] [5064] [?v=0] [GET /css/button.css?v=0 HTTP/1.1] [-] [200] [200] [0] [/css/button.css] [hostname.domain] [hostname.domain] [+] [1540] [1138] [" http://hostname.domain/index.html "] ["Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0"]

For example, i create test.log that contains:
[2015-06-09 16:47:34 830 NOVT]

and test.rb:
rule=:[%date:date-iso% %time:time-24hr% %microsec:number% %timezone:char-to:]%

If i try i get:
$?lognormalizer -r test.rb -e json? < test.log 
{ "originalmsg": "[2015-06-09 16:47:34 830 NOVT]", "unparsed-data": "]" }

How to parse data between [ and ] ?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20150610/578b189e/attachment.html>

From chrisp.schafer at gmail.com  Wed Jun 10 08:12:07 2015
From: chrisp.schafer at gmail.com (Chris Schafer)
Date: Wed, 10 Jun 2015 06:12:07 +0000
Subject: [Lognorm] Escape [ and ]
In-Reply-To: <1433916627.609174282@f94.i.mail.ru>
References: <1433916627.609174282@f94.i.mail.ru>
Message-ID: <CAObX7j5opB-QN9Ua9-J0Drq+S9pG0j3GpUJjqj6whPemSmH2Tg@mail.gmail.com>

You are almost doing it right, just add a ] at the end - you are parsing TO
the ], not including it.
rule=:[%date:date-iso% %time:time-24hr% %microsec:number%
%timezone:char-to:]%]
On Tue, Jun 9, 2015 at 11:10 PM ?????? ??????? ??????? <fvl at mail.ru> wrote:

> Hi!
> I want to log all from apache. I use custom log in apache that looks like:
>
> [2015-06-09 18:27:07 197 NOVT] [192.168.1.67] [192.168.1.67]
> [192.168.1.254] [818] [/var/www/host/css/button.css] [192.168.1.67]
> [HTTP/1.1] [1] [-] [GET] [5064] [?v=0] [GET /css/button.css?v=0 HTTP/1.1]
> [-] [200] [200] [0] [/css/button.css] [hostname.domain] [hostname.domain]
> [+] [1540] [1138] ["http://hostname.domain/index.html"] ["Mozilla/5.0
> (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0"]
>
> For example, i create test.log that contains:
> [2015-06-09 16:47:34 830 NOVT]
>
> and test.rb:
> rule=:[%date:date-iso% %time:time-24hr% %microsec:number%
> %timezone:char-to:]%
>
> If i try i get:
> $ lognormalizer -r test.rb -e json  < test.log
> { "originalmsg": "[2015-06-09 16:47:34 830 NOVT]", "unparsed-data": "]" }
>
> How to parse data between [ and ] ?
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20150610/db8901ce/attachment.html>

From david at lang.hm  Wed Jun 10 08:13:16 2015
From: david at lang.hm (David Lang)
Date: Tue, 9 Jun 2015 23:13:16 -0700 (PDT)
Subject: [Lognorm] Escape [ and ]
In-Reply-To: <1433916627.609174282@f94.i.mail.ru>
References: <1433916627.609174282@f94.i.mail.ru>
Message-ID: <alpine.DEB.2.02.1506092312110.24535@nftneq.ynat.uz>

On Wed, 10 Jun 2015, ?????? ??????? ??????? wrote:

>
> Hi!
> I want to log all from apache. I use custom log in apache that looks like:
>
> [2015-06-09 18:27:07 197 NOVT] [192.168.1.67] [192.168.1.67] [192.168.1.254] [818] [/var/www/host/css/button.css] [192.168.1.67] [HTTP/1.1] [1] [-] [GET] [5064] [?v=0] [GET /css/button.css?v=0 HTTP/1.1] [-] [200] [200] [0] [/css/button.css] [hostname.domain] [hostname.domain] [+] [1540] [1138] [" http://hostname.domain/index.html "] ["Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0"]
>
> For example, i create test.log that contains:
> [2015-06-09 16:47:34 830 NOVT]
>
> and test.rb:
> rule=:[%date:date-iso% %time:time-24hr% %microsec:number% %timezone:char-to:]%
>
> If i try i get:
> $?lognormalizer -r test.rb -e json? < test.log 
> { "originalmsg": "[2015-06-09 16:47:34 830 NOVT]", "unparsed-data": "]" }
>
> How to parse data between [ and ] ?

you almost have it correct. The only thing that you are missing is that char-to 
doesn't 'consume' the matching character, so your rule would need to be:

rule=:[%date:date-iso% %time:time-24hr% %microsec:number% %timezone:char-to:]%]

David Lang
-------------- next part --------------
_______________________________________________
Lognorm mailing list
Lognorm at lists.adiscon.com
http://lists.adiscon.net/mailman/listinfo/lognorm

From fvl at mail.ru  Wed Jun 10 08:13:19 2015
From: fvl at mail.ru (=?UTF-8?B?0KTQsNC00LXQtdCyINCS0LjRgtCw0LvQuNC5INCb0YzQstC+0LLQuNGH?=)
Date: Wed, 10 Jun 2015 09:13:19 +0300
Subject: [Lognorm] =?utf-8?q?Escape_=5B_and_=5D?=
In-Reply-To: <CAObX7j5opB-QN9Ua9-J0Drq+S9pG0j3GpUJjqj6whPemSmH2Tg@mail.gmail.com>
References: <1433916627.609174282@f94.i.mail.ru>
	<CAObX7j5opB-QN9Ua9-J0Drq+S9pG0j3GpUJjqj6whPemSmH2Tg@mail.gmail.com>
Message-ID: <1433916799.976560542@f364.i.mail.ru>

 Thanks!
This works!
Big thanks!


?????, 10 ???? 2015, 6:12 UTC ?? Chris Schafer <chrisp.schafer at gmail.com>:
>You are almost doing it right, just add a ] at the end - you are parsing TO the ], not including it. 
>rule=:[%date:date-iso% %time:time-24hr% %microsec:number% %timezone:char-to:]%]
>On Tue, Jun 9, 2015 at 11:10 PM ?????? ??????? ??????? < fvl at mail.ru > wrote:
>>Hi!
>>I want to log all from apache. I use custom log in apache that looks like:
>>
>>[2015-06-09 18:27:07 197 NOVT] [192.168.1.67] [192.168.1.67] [192.168.1.254] [818] [/var/www/host/css/button.css] [192.168.1.67] [HTTP/1.1] [1] [-] [GET] [5064] [?v=0] [GET /css/button.css?v=0 HTTP/1.1] [-] [200] [200] [0] [/css/button.css] [hostname.domain] [hostname.domain] [+] [1540] [1138] [" http://hostname.domain/index.html "] ["Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0"]
>>
>>For example, i create test.log that contains:
>>[2015-06-09 16:47:34 830 NOVT]
>>
>>and test.rb:
>>rule=:[%date:date-iso% %time:time-24hr% %microsec:number% %timezone:char-to:]%
>>
>>If i try i get:
>>$?lognormalizer -r test.rb -e json? < test.log 
>>{ "originalmsg": "[2015-06-09 16:47:34 830 NOVT]", "unparsed-data": "]" }
>>
>>How to parse data between [ and ] ?
>>_______________________________________________
>>Lognorm mailing list
>>Lognorm at lists.adiscon.com
>>http://lists.adiscon.net/mailman/listinfo/lognorm

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20150610/3bdef916/attachment.html>

From chrisp.schafer at gmail.com  Wed Jun 10 08:15:42 2015
From: chrisp.schafer at gmail.com (Chris Schafer)
Date: Wed, 10 Jun 2015 06:15:42 +0000
Subject: [Lognorm] Escape [ and ]
In-Reply-To: <alpine.DEB.2.02.1506092312110.24535@nftneq.ynat.uz>
References: <1433916627.609174282@f94.i.mail.ru>
	<alpine.DEB.2.02.1506092312110.24535@nftneq.ynat.uz>
Message-ID: <CAObX7j5uePVQoxe77zs+kcaUqe74E1YhWQCzWAO5ugPMqRU-0Q@mail.gmail.com>

Glad I can help. I spent way too much time learning this stuff and I don't
get to use it often enough :)
On Tue, Jun 9, 2015 at 11:13 PM David Lang <david at lang.hm> wrote:

> On Wed, 10 Jun 2015, ?????? ??????? ??????? wrote:
>
> >
> > Hi!
> > I want to log all from apache. I use custom log in apache that looks
> like:
> >
> > [2015-06-09 18:27:07 197 NOVT] [192.168.1.67] [192.168.1.67]
> [192.168.1.254] [818] [/var/www/host/css/button.css] [192.168.1.67]
> [HTTP/1.1] [1] [-] [GET] [5064] [?v=0] [GET /css/button.css?v=0 HTTP/1.1]
> [-] [200] [200] [0] [/css/button.css] [hostname.domain] [hostname.domain]
> [+] [1540] [1138] [" http://hostname.domain/index.html "] ["Mozilla/5.0
> (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0"]
> >
> > For example, i create test.log that contains:
> > [2015-06-09 16:47:34 830 NOVT]
> >
> > and test.rb:
> > rule=:[%date:date-iso% %time:time-24hr% %microsec:number%
> %timezone:char-to:]%
> >
> > If i try i get:
> > $ lognormalizer -r test.rb -e json  < test.log
> > { "originalmsg": "[2015-06-09 16:47:34 830 NOVT]", "unparsed-data": "]" }
> >
> > How to parse data between [ and ] ?
>
> you almost have it correct. The only thing that you are missing is that
> char-to
> doesn't 'consume' the matching character, so your rule would need to be:
>
> rule=:[%date:date-iso% %time:time-24hr% %microsec:number%
> %timezone:char-to:]%]
>
> David Lang
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20150610/108b51c4/attachment-0001.html>

From fvl at mail.ru  Wed Jun 10 08:21:59 2015
From: fvl at mail.ru (=?UTF-8?B?0KTQsNC00LXQtdCyINCS0LjRgtCw0LvQuNC5INCb0YzQstC+0LLQuNGH?=)
Date: Wed, 10 Jun 2015 09:21:59 +0300
Subject: [Lognorm] =?utf-8?q?Escape_=5B_and_=5D?=
In-Reply-To: <CAObX7j5uePVQoxe77zs+kcaUqe74E1YhWQCzWAO5ugPMqRU-0Q@mail.gmail.com>
References: <1433916627.609174282@f94.i.mail.ru>
	<alpine.DEB.2.02.1506092312110.24535@nftneq.ynat.uz>
	<CAObX7j5uePVQoxe77zs+kcaUqe74E1YhWQCzWAO5ugPMqRU-0Q@mail.gmail.com>
Message-ID: <1433917319.789235369@f281.i.mail.ru>

 Also, i did not find how to get IPv6 and IPv4 string
For example:

IP: 2607:f8b0:4000:807::1006:
IP: 64.233.160.106

This all of google.com


?????, 10 ???? 2015, 6:15 UTC ?? Chris Schafer <chrisp.schafer at gmail.com>:
>Glad I can help. I spent way too much time learning this stuff and I don't get to use it often enough :)
>On Tue, Jun 9, 2015 at 11:13 PM David Lang < david at lang.hm > wrote:
>>On Wed, 10 Jun 2015, ?????? ??????? ??????? wrote:
>>
>>>
>>> Hi!
>>> I want to log all from apache. I use custom log in apache that looks like:
>>>
>>> [2015-06-09 18:27:07 197 NOVT] [192.168.1.67] [192.168.1.67] [192.168.1.254] [818] [/var/www/host/css/button.css] [192.168.1.67] [HTTP/1.1] [1] [-] [GET] [5064] [?v=0] [GET /css/button.css?v=0 HTTP/1.1] [-] [200] [200] [0] [/css/button.css] [hostname.domain] [hostname.domain] [+] [1540] [1138] ["  http://hostname.domain/index.html "] ["Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0"]
>>>
>>> For example, i create test.log that contains:
>>> [2015-06-09 16:47:34 830 NOVT]
>>>
>>> and test.rb:
>>> rule=:[%date:date-iso% %time:time-24hr% %microsec:number% %timezone:char-to:]%
>>>
>>> If i try i get:
>>> $?lognormalizer -r test.rb -e json? < test.log
>>> { "originalmsg": "[2015-06-09 16:47:34 830 NOVT]", "unparsed-data": "]" }
>>>
>>> How to parse data between [ and ] ?
>>
>>you almost have it correct. The only thing that you are missing is that char-to
>>doesn't 'consume' the matching character, so your rule would need to be:
>>
>>rule=:[%date:date-iso% %time:time-24hr% %microsec:number% %timezone:char-to:]%]
>>
>>David Lang
>>_______________________________________________
>>Lognorm mailing list
>>Lognorm at lists.adiscon.com
>>http://lists.adiscon.net/mailman/listinfo/lognorm
>>_______________________________________________
>>Lognorm mailing list
>>Lognorm at lists.adiscon.com
>>http://lists.adiscon.net/mailman/listinfo/lognorm

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20150610/4650e2ce/attachment.html>

From fvl at mail.ru  Wed Jun 10 08:59:03 2015
From: fvl at mail.ru (=?UTF-8?B?0KTQsNC00LXQtdCyINCS0LjRgtCw0LvQuNC5INCb0YzQstC+0LLQuNGH?=)
Date: Wed, 10 Jun 2015 09:59:03 +0300
Subject: [Lognorm] =?utf-8?q?Escape_=5B_and_=5D?=
In-Reply-To: <1433917319.789235369@f281.i.mail.ru>
References: <1433916627.609174282@f94.i.mail.ru>
	<CAObX7j5uePVQoxe77zs+kcaUqe74E1YhWQCzWAO5ugPMqRU-0Q@mail.gmail.com>
	<1433917319.789235369@f281.i.mail.ru>
Message-ID: <1433919543.213874359@f223.i.mail.ru>

 And, probably, there is mismatch:

$ cat num.txt 
num: 42

$ cat num.rb
rule=:num: %answer:number%

$lognormalizer -r num.rb -e json < num.txt 
{ "answer": "42" }

But if i read on  https://en.wikipedia.org/wiki/JSON

JSON's basic types are:
*  Number: a signed decimal number that may contain a fractional part and may use exponential  E notation . JSON does not allow non-numbers like  NaN , nor does it make any distinction between integer and floating-point. (Even though JavaScript uses a  double-precision floating-point format for all its numeric values, other languages implementing JSON may encode numbers differently)
*  String : a sequence of zero or more  Unicode characters. Strings are delimited with double-quotation marks and support a backslash  escaping syntax.

So, i think lognorm return wrong JSON. This is must be without double quotes:
{ "answer": 42 }

Is this expected behavior?

>Also, i did not find how to get IPv6 and IPv4 string
>For example:
>
>IP: 2607:f8b0:4000:807::1006:
>IP: 64.233.160.106
>
>This all of google.com
>
>
>?????, 10 ???? 2015, 6:15 UTC ?? Chris Schafer <chrisp.schafer at gmail.com>:
>>Glad I can help. I spent way too much time learning this stuff and I don't get to use it often enough :)
>>On Tue, Jun 9, 2015 at 11:13 PM David Lang < david at lang.hm > wrote:
>>>On Wed, 10 Jun 2015, ?????? ??????? ??????? wrote:
>>>
>>>>
>>>> Hi!
>>>> I want to log all from apache. I use custom log in apache that looks like:
>>>>
>>>> [2015-06-09 18:27:07 197 NOVT] [192.168.1.67] [192.168.1.67] [192.168.1.254] [818] [/var/www/host/css/button.css] [192.168.1.67] [HTTP/1.1] [1] [-] [GET] [5064] [?v=0] [GET /css/button.css?v=0 HTTP/1.1] [-] [200] [200] [0] [/css/button.css] [hostname.domain] [hostname.domain] [+] [1540] [1138] ["  http://hostname.domain/index.html "] ["Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0"]
>>>>
>>>> For example, i create test.log that contains:
>>>> [2015-06-09 16:47:34 830 NOVT]
>>>>
>>>> and test.rb:
>>>> rule=:[%date:date-iso% %time:time-24hr% %microsec:number% %timezone:char-to:]%
>>>>
>>>> If i try i get:
>>>> $?lognormalizer -r test.rb -e json? < test.log
>>>> { "originalmsg": "[2015-06-09 16:47:34 830 NOVT]", "unparsed-data": "]" }
>>>>
>>>> How to parse data between [ and ] ?
>>>
>>>you almost have it correct. The only thing that you are missing is that char-to
>>>doesn't 'consume' the matching character, so your rule would need to be:
>>>
>>>rule=:[%date:date-iso% %time:time-24hr% %microsec:number% %timezone:char-to:]%]
>>>
>>>David Lang
>>>_______________________________________________
>>>Lognorm mailing list
>>>Lognorm at lists.adiscon.com
>>>http://lists.adiscon.net/mailman/listinfo/lognorm
>>>_______________________________________________
>>>Lognorm mailing list
>>>Lognorm at lists.adiscon.com
>>>http://lists.adiscon.net/mailman/listinfo/lognorm
>
>_______________________________________________
>Lognorm mailing list
>Lognorm at lists.adiscon.com
>http://lists.adiscon.net/mailman/listinfo/lognorm

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20150610/e32eaac4/attachment.html>

From chrisp.schafer at gmail.com  Wed Jun 10 09:00:22 2015
From: chrisp.schafer at gmail.com (Chris Schafer)
Date: Wed, 10 Jun 2015 07:00:22 +0000
Subject: [Lognorm] Escape [ and ]
In-Reply-To: <1433919543.213874359@f223.i.mail.ru>
References: <1433916627.609174282@f94.i.mail.ru>
	<CAObX7j5uePVQoxe77zs+kcaUqe74E1YhWQCzWAO5ugPMqRU-0Q@mail.gmail.com>
	<1433917319.789235369@f281.i.mail.ru>
	<1433919543.213874359@f223.i.mail.ru>
Message-ID: <CAObX7j54r4meo+XuuF88VgApLDAWP1fFnqK=rTTLDtEXRZq_5Q@mail.gmail.com>

I believe it ends up taking everything as a string, but I could be wrong.
On Tue, Jun 9, 2015 at 11:59 PM ?????? ??????? ??????? <fvl at mail.ru> wrote:

> And, probably, there is mismatch:
>
> $ cat num.txt
> num: 42
>
> $ cat num.rb
> rule=:num: %answer:number%
>
> $lognormalizer -r num.rb -e json < num.txt
> { "answer": "42" }
>
> But if i read on https://en.wikipedia.org/wiki/JSON
>
> JSON's basic types are:
>
>    - Number: a signed decimal number that may contain a fractional part
>    and may use exponential E notation
>    <https://en.wikipedia.org/wiki/E_notation>. JSON does not allow
>    non-numbers like NaN <https://en.wikipedia.org/wiki/NaN>, nor does it
>    make any distinction between integer and floating-point. (Even though
>    JavaScript uses a double-precision floating-point format
>    <https://en.wikipedia.org/wiki/Double-precision_floating-point_format>
>    for all its numeric values, other languages implementing JSON may encode
>    numbers differently)
>    - String <https://en.wikipedia.org/wiki/String_(computer_science)>: a
>    sequence of zero or more Unicode
>    <https://en.wikipedia.org/wiki/Unicode> characters. Strings are
>    delimited with double-quotation marks and support a backslash escaping
>    <https://en.wikipedia.org/wiki/Escape_character> syntax.
>
>
> So, i think lognorm return wrong JSON. This is must be without double
> quotes:
> { "answer": 42 }
>
> Is this expected behavior?
>
> <https://en.wikipedia.org/wiki/JSON>
>
>   Also, i did not find how to get IPv6 and IPv4 string
> For example:
>
> IP: 2607:f8b0:4000:807::1006:
> IP: 64.233.160.106
>
> This all of google.com
>
>
> ?????, 10 ???? 2015, 6:15 UTC ?? Chris Schafer <chrisp.schafer at gmail.com>:
>
>   Glad I can help. I spent way too much time learning this stuff and I
> don't get to use it often enough :)
> On Tue, Jun 9, 2015 at 11:13 PM David Lang <david at lang.hm
> <//e.mail.ru/compose/?mailto=mailto%3adavid at lang.hm>> wrote:
>
> On Wed, 10 Jun 2015, ?????? ??????? ??????? wrote:
>
> >
> > Hi!
> > I want to log all from apache. I use custom log in apache that looks
> like:
> >
> > [2015-06-09 18:27:07 197 NOVT] [192.168.1.67] [192.168.1.67]
> [192.168.1.254] [818] [/var/www/host/css/button.css] [192.168.1.67]
> [HTTP/1.1] [1] [-] [GET] [5064] [?v=0] [GET /css/button.css?v=0 HTTP/1.1]
> [-] [200] [200] [0] [/css/button.css] [hostname.domain] [hostname.domain]
> [+] [1540] [1138] [" http://hostname.domain/index.html "] ["Mozilla/5.0
> (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0"]
> >
> > For example, i create test.log that contains:
> > [2015-06-09 16:47:34 830 NOVT]
> >
> > and test.rb:
> > rule=:[%date:date-iso% %time:time-24hr% %microsec:number%
> %timezone:char-to:]%
> >
> > If i try i get:
> > $ lognormalizer -r test.rb -e json  < test.log
> > { "originalmsg": "[2015-06-09 16:47:34 830 NOVT]", "unparsed-data": "]" }
> >
> > How to parse data between [ and ] ?
>
> you almost have it correct. The only thing that you are missing is that
> char-to
> doesn't 'consume' the matching character, so your rule would need to be:
>
> rule=:[%date:date-iso% %time:time-24hr% %microsec:number%
> %timezone:char-to:]%]
>
> David Lang
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> <//e.mail.ru/compose/?mailto=mailto%3aLognorm at lists.adiscon.com>
> http://lists.adiscon.net/mailman/listinfo/lognorm
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> <//e.mail.ru/compose/?mailto=mailto%3aLognorm at lists.adiscon.com>
> http://lists.adiscon.net/mailman/listinfo/lognorm
>
>
>  _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm
>
>
> <https://en.wikipedia.org/wiki/JSON>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20150610/7cc1f358/attachment-0001.html>

From fvl at mail.ru  Wed Jun 10 09:03:59 2015
From: fvl at mail.ru (=?UTF-8?B?0KTQsNC00LXQtdCyINCS0LjRgtCw0LvQuNC5INCb0YzQstC+0LLQuNGH?=)
Date: Wed, 10 Jun 2015 10:03:59 +0300
Subject: [Lognorm] =?utf-8?q?Escape_=5B_and_=5D?=
In-Reply-To: <CAObX7j54r4meo+XuuF88VgApLDAWP1fFnqK=rTTLDtEXRZq_5Q@mail.gmail.com>
References: <1433916627.609174282@f94.i.mail.ru>
	<1433919543.213874359@f223.i.mail.ru>
	<CAObX7j54r4meo+XuuF88VgApLDAWP1fFnqK=rTTLDtEXRZq_5Q@mail.gmail.com>
Message-ID: <1433919839.244263580@f301.i.mail.ru>

 I think this is wrong. For example: unix timestamp. This is always number.


?????, 10 ???? 2015, 7:00 UTC ?? Chris Schafer <chrisp.schafer at gmail.com>:
>I believe it ends up taking everything as a string, but I could be wrong. 
>On Tue, Jun 9, 2015 at 11:59 PM ?????? ??????? ??????? < fvl at mail.ru > wrote:
>>And, probably, there is mismatch:
>>
>>$ cat num.txt 
>>num: 42
>>
>>$ cat num.rb
>>rule=:num: %answer:number%
>>
>>$lognormalizer -r num.rb -e json < num.txt 
>>{ "answer": "42" }
>>
>>But if i read on  https://en.wikipedia.org/wiki/JSON
>>
>>JSON's basic types are:
>>*  Number: a signed decimal number that may contain a fractional part and may use exponential  E notation . JSON does not allow non-numbers like  NaN , nor does it make any distinction between integer and floating-point. (Even though JavaScript uses a  double-precision floating-point format for all its numeric values, other languages implementing JSON may encode numbers differently)
>>*  String : a sequence of zero or more  Unicode characters. Strings are delimited with double-quotation marks and support a backslash  escaping syntax.
>>
>>So, i think lognorm return wrong JSON. This is must be without double quotes:
>>{ "answer": 42 }
>>
>>Is this expected behavior?
>>
>>>Also, i did not find how to get IPv6 and IPv4 string
>>>For example:
>>>
>>>IP: 2607:f8b0:4000:807::1006:
>>>IP: 64.233.160.106
>>>
>>>This all of google.com
>>>
>>>
>>>?????, 10 ???? 2015, 6:15 UTC ?? Chris Schafer <chrisp.schafer at gmail.com>:
>>>>Glad I can help. I spent way too much time learning this stuff and I don't get to use it often enough :)
>>>>On Tue, Jun 9, 2015 at 11:13 PM David Lang < david at lang.hm > wrote:
>>>>>On Wed, 10 Jun 2015, ?????? ??????? ??????? wrote:
>>>>>
>>>>>>
>>>>>> Hi!
>>>>>> I want to log all from apache. I use custom log in apache that looks like:
>>>>>>
>>>>>> [2015-06-09 18:27:07 197 NOVT] [192.168.1.67] [192.168.1.67] [192.168.1.254] [818] [/var/www/host/css/button.css] [192.168.1.67] [HTTP/1.1] [1] [-] [GET] [5064] [?v=0] [GET /css/button.css?v=0 HTTP/1.1] [-] [200] [200] [0] [/css/button.css] [hostname.domain] [hostname.domain] [+] [1540] [1138] ["  http://hostname.domain/index.html "] ["Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0"]
>>>>>>
>>>>>> For example, i create test.log that contains:
>>>>>> [2015-06-09 16:47:34 830 NOVT]
>>>>>>
>>>>>> and test.rb:
>>>>>> rule=:[%date:date-iso% %time:time-24hr% %microsec:number% %timezone:char-to:]%
>>>>>>
>>>>>> If i try i get:
>>>>>> $?lognormalizer -r test.rb -e json? < test.log
>>>>>> { "originalmsg": "[2015-06-09 16:47:34 830 NOVT]", "unparsed-data": "]" }
>>>>>>
>>>>>> How to parse data between [ and ] ?
>>>>>
>>>>>you almost have it correct. The only thing that you are missing is that char-to
>>>>>doesn't 'consume' the matching character, so your rule would need to be:
>>>>>
>>>>>rule=:[%date:date-iso% %time:time-24hr% %microsec:number% %timezone:char-to:]%]
>>>>>
>>>>>David Lang
>>>>>_______________________________________________
>>>>>Lognorm mailing list
>>>>>Lognorm at lists.adiscon.com
>>>>>http://lists.adiscon.net/mailman/listinfo/lognorm
>>>>>_______________________________________________
>>>>>Lognorm mailing list
>>>>>Lognorm at lists.adiscon.com
>>>>>http://lists.adiscon.net/mailman/listinfo/lognorm
>>>
>>>_______________________________________________
>>>Lognorm mailing list
>>>Lognorm at lists.adiscon.com
>>>http://lists.adiscon.net/mailman/listinfo/lognorm
>>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20150610/dc6dd84f/attachment.html>

From fvl at mail.ru  Wed Jun 10 09:11:05 2015
From: fvl at mail.ru (=?UTF-8?B?0KTQsNC00LXQtdCyINCS0LjRgtCw0LvQuNC5INCb0YzQstC+0LLQuNGH?=)
Date: Wed, 10 Jun 2015 10:11:05 +0300
Subject: [Lognorm] =?utf-8?q?Escape_=5B_and_=5D?=
In-Reply-To: <1433919839.244263580@f301.i.mail.ru>
References: <1433916627.609174282@f94.i.mail.ru>
	<CAObX7j54r4meo+XuuF88VgApLDAWP1fFnqK=rTTLDtEXRZq_5Q@mail.gmail.com>
	<1433919839.244263580@f301.i.mail.ru>
Message-ID: <1433920265.312924674@f164.i.mail.ru>


Lets look into this example:

$ cat test.php
<?php
$json_wrong = '{ "answer": "42" }';
$json_correct = '{ "answer": 42 }';
var_dump(json_decode($json_wrong));
var_dump(json_decode($json_correct));
?>

$ php test.php
object(stdClass)#1 (1) {
? ["answer"]=>
? string(2) "42"
}
object(stdClass)#1 (1) {
? ["answer"]=>
? int(42)
}

?????, 10 ???? 2015, 10:03 +03:00 ?? ?????? ??????? ??????? <fvl at mail.ru>:
>I think this is wrong. For example: unix timestamp. This is always number.
>
>
>?????, 10 ???? 2015, 7:00 UTC ?? Chris Schafer <chrisp.schafer at gmail.com>:
>>I believe it ends up taking everything as a string, but I could be wrong. 
>>On Tue, Jun 9, 2015 at 11:59 PM ?????? ??????? ??????? < fvl at mail.ru > wrote:
>>>And, probably, there is mismatch:
>>>
>>>$ cat num.txt 
>>>num: 42
>>>
>>>$ cat num.rb
>>>rule=:num: %answer:number%
>>>
>>>$lognormalizer -r num.rb -e json < num.txt 
>>>{ "answer": "42" }
>>>
>>>But if i read on  https://en.wikipedia.org/wiki/JSON
>>>
>>>JSON's basic types are:
>>>*  Number: a signed decimal number that may contain a fractional part and may use exponential  E notation . JSON does not allow non-numbers like  NaN , nor does it make any distinction between integer and floating-point. (Even though JavaScript uses a  double-precision floating-point format for all its numeric values, other languages implementing JSON may encode numbers differently)
>>>*  String : a sequence of zero or more  Unicode characters. Strings are delimited with double-quotation marks and support a backslash  escaping syntax.
>>>
>>>So, i think lognorm return wrong JSON. This is must be without double quotes:
>>>{ "answer": 42 }
>>>
>>>Is this expected behavior?
>>>
>>>>Also, i did not find how to get IPv6 and IPv4 string
>>>>For example:
>>>>
>>>>IP: 2607:f8b0:4000:807::1006:
>>>>IP: 64.233.160.106
>>>>
>>>>This all of google.com
>>>>
>>>>
>>>>?????, 10 ???? 2015, 6:15 UTC ?? Chris Schafer <chrisp.schafer at gmail.com>:
>>>>>Glad I can help. I spent way too much time learning this stuff and I don't get to use it often enough :)
>>>>>On Tue, Jun 9, 2015 at 11:13 PM David Lang < david at lang.hm > wrote:
>>>>>>On Wed, 10 Jun 2015, ?????? ??????? ??????? wrote:
>>>>>>
>>>>>>>
>>>>>>> Hi!
>>>>>>> I want to log all from apache. I use custom log in apache that looks like:
>>>>>>>
>>>>>>> [2015-06-09 18:27:07 197 NOVT] [192.168.1.67] [192.168.1.67] [192.168.1.254] [818] [/var/www/host/css/button.css] [192.168.1.67] [HTTP/1.1] [1] [-] [GET] [5064] [?v=0] [GET /css/button.css?v=0 HTTP/1.1] [-] [200] [200] [0] [/css/button.css] [hostname.domain] [hostname.domain] [+] [1540] [1138] ["  http://hostname.domain/index.html "] ["Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0"]
>>>>>>>
>>>>>>> For example, i create test.log that contains:
>>>>>>> [2015-06-09 16:47:34 830 NOVT]
>>>>>>>
>>>>>>> and test.rb:
>>>>>>> rule=:[%date:date-iso% %time:time-24hr% %microsec:number% %timezone:char-to:]%
>>>>>>>
>>>>>>> If i try i get:
>>>>>>> $?lognormalizer -r test.rb -e json? < test.log
>>>>>>> { "originalmsg": "[2015-06-09 16:47:34 830 NOVT]", "unparsed-data": "]" }
>>>>>>>
>>>>>>> How to parse data between [ and ] ?
>>>>>>
>>>>>>you almost have it correct. The only thing that you are missing is that char-to
>>>>>>doesn't 'consume' the matching character, so your rule would need to be:
>>>>>>
>>>>>>rule=:[%date:date-iso% %time:time-24hr% %microsec:number% %timezone:char-to:]%]
>>>>>>
>>>>>>David Lang
>>>>>>_______________________________________________
>>>>>>Lognorm mailing list
>>>>>>Lognorm at lists.adiscon.com
>>>>>>http://lists.adiscon.net/mailman/listinfo/lognorm
>>>>>>_______________________________________________
>>>>>>Lognorm mailing list
>>>>>>Lognorm at lists.adiscon.com
>>>>>>http://lists.adiscon.net/mailman/listinfo/lognorm
>>>>
>>>>_______________________________________________
>>>>Lognorm mailing list
>>>>Lognorm at lists.adiscon.com
>>>>http://lists.adiscon.net/mailman/listinfo/lognorm
>>>
>
>_______________________________________________
>Lognorm mailing list
>Lognorm at lists.adiscon.com
>http://lists.adiscon.net/mailman/listinfo/lognorm

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20150610/b9eacedb/attachment-0001.html>

From david at lang.hm  Wed Jun 10 19:39:34 2015
From: david at lang.hm (David Lang)
Date: Wed, 10 Jun 2015 10:39:34 -0700 (PDT)
Subject: [Lognorm] Escape [ and ]
In-Reply-To: <1433919543.213874359@f223.i.mail.ru>
References: <1433916627.609174282@f94.i.mail.ru>
	<CAObX7j5uePVQoxe77zs+kcaUqe74E1YhWQCzWAO5ugPMqRU-0Q@mail.gmail.com>
	<1433917319.789235369@f281.i.mail.ru>
	<1433919543.213874359@f223.i.mail.ru>
Message-ID: <alpine.DEB.2.02.1506101038180.17605@nftneq.ynat.uz>

On Wed, 10 Jun 2015, ?????? ??????? ??????? wrote:

> And, probably, there is mismatch:
>
> $ cat num.txt 
> num: 42
>
> $ cat num.rb
> rule=:num: %answer:number%
>
> $lognormalizer -r num.rb -e json < num.txt 
> { "answer": "42" }
>
> But if i read on  https://en.wikipedia.org/wiki/JSON
>
> JSON's basic types are:
> *  Number: a signed decimal number that may contain a fractional part and may use exponential  E notation . JSON does not allow non-numbers like  NaN , nor does it make any distinction between integer and floating-point. (Even though JavaScript uses a  double-precision floating-point format for all its numeric values, other languages implementing JSON may encode numbers differently)
> *  String : a sequence of zero or more  Unicode characters. Strings are delimited with double-quotation marks and support a backslash  escaping syntax.
>
> So, i think lognorm return wrong JSON. This is must be without double quotes:
> { "answer": 42 }
>
> Is this expected behavior?

this is the current behavior. The unreleased version in git has a new function 
'interpret' that can convert strings to numbers among other things.


>> Also, i did not find how to get IPv6 and IPv4 string
>> For example:
>>
>> IP: 2607:f8b0:4000:807::1006:
>> IP: 64.233.160.106

there is an IPv4 type, but not a IPv6 type, it probably should be added.

David Lang

>> This all of google.com
>>
>>
>> ?????, 10 ???? 2015, 6:15 UTC ?? Chris Schafer <chrisp.schafer at gmail.com>:
>>> Glad I can help. I spent way too much time learning this stuff and I don't get to use it often enough :)
>>> On Tue, Jun 9, 2015 at 11:13 PM David Lang < david at lang.hm > wrote:
>>>> On Wed, 10 Jun 2015, ?????? ??????? ??????? wrote:
>>>>
>>>>>
>>>>> Hi!
>>>>> I want to log all from apache. I use custom log in apache that looks like:
>>>>>
>>>>> [2015-06-09 18:27:07 197 NOVT] [192.168.1.67] [192.168.1.67] [192.168.1.254] [818] [/var/www/host/css/button.css] [192.168.1.67] [HTTP/1.1] [1] [-] [GET] [5064] [?v=0] [GET /css/button.css?v=0 HTTP/1.1] [-] [200] [200] [0] [/css/button.css] [hostname.domain] [hostname.domain] [+] [1540] [1138] ["  http://hostname.domain/index.html "] ["Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0"]
>>>>>
>>>>> For example, i create test.log that contains:
>>>>> [2015-06-09 16:47:34 830 NOVT]
>>>>>
>>>>> and test.rb:
>>>>> rule=:[%date:date-iso% %time:time-24hr% %microsec:number% %timezone:char-to:]%
>>>>>
>>>>> If i try i get:
>>>>> $?lognormalizer -r test.rb -e json? < test.log
>>>>> { "originalmsg": "[2015-06-09 16:47:34 830 NOVT]", "unparsed-data": "]" }
>>>>>
>>>>> How to parse data between [ and ] ?
>>>>
>>>> you almost have it correct. The only thing that you are missing is that char-to
>>>> doesn't 'consume' the matching character, so your rule would need to be:
>>>>
>>>> rule=:[%date:date-iso% %time:time-24hr% %microsec:number% %timezone:char-to:]%]
>>>>
>>>> David Lang
>>>> _______________________________________________
>>>> Lognorm mailing list
>>>> Lognorm at lists.adiscon.com
>>>> http://lists.adiscon.net/mailman/listinfo/lognorm
>>>> _______________________________________________
>>>> Lognorm mailing list
>>>> Lognorm at lists.adiscon.com
>>>> http://lists.adiscon.net/mailman/listinfo/lognorm
>>
>> _______________________________________________
>> Lognorm mailing list
>> Lognorm at lists.adiscon.com
>> http://lists.adiscon.net/mailman/listinfo/lognorm
-------------- next part --------------
_______________________________________________
Lognorm mailing list
Lognorm at lists.adiscon.com
http://lists.adiscon.net/mailman/listinfo/lognorm

From rgerhards at hq.adiscon.com  Thu Jun 11 07:30:40 2015
From: rgerhards at hq.adiscon.com (Rainer Gerhards)
Date: Thu, 11 Jun 2015 07:30:40 +0200
Subject: [Lognorm] Escape [ and ]
In-Reply-To: <1433919839.244263580@f301.i.mail.ru>
References: <1433916627.609174282@f94.i.mail.ru>
	<1433919543.213874359@f223.i.mail.ru>
	<CAObX7j54r4meo+XuuF88VgApLDAWP1fFnqK=rTTLDtEXRZq_5Q@mail.gmail.com>
	<1433919839.244263580@f301.i.mail.ru>
Message-ID: <CADk+mPAnRjBdjzicN4Cwws_TQ690_M6h29eQ_VWT0Fj8Om06Qg@mail.gmail.com>

lognorm processes log messages and log messages are by definition
always string and so lognorm always returns string in the initial
implementation. It is right that one can argue we should return other
types if they are obvious and that's a good argument, but it would
break compatibility at least for existing types. There is also a
reinterpret capability.

Rainer

2015-06-10 9:03 GMT+02:00 ?????? ??????? ??????? <fvl at mail.ru>:
> I think this is wrong. For example: unix timestamp. This is always number.
>
>
> ?????, 10 ???? 2015, 7:00 UTC ?? Chris Schafer <chrisp.schafer at gmail.com>:
>
> I believe it ends up taking everything as a string, but I could be wrong.
> On Tue, Jun 9, 2015 at 11:59 PM ?????? ??????? ??????? <fvl at mail.ru> wrote:
>
> And, probably, there is mismatch:
>
> $ cat num.txt
> num: 42
>
> $ cat num.rb
> rule=:num: %answer:number%
>
> $lognormalizer -r num.rb -e json < num.txt
> { "answer": "42" }
>
> But if i read on https://en.wikipedia.org/wiki/JSON
>
> JSON's basic types are:
>
> Number: a signed decimal number that may contain a fractional part and may
> use exponential E notation. JSON does not allow non-numbers like NaN, nor
> does it make any distinction between integer and floating-point. (Even
> though JavaScript uses a double-precision floating-point format for all its
> numeric values, other languages implementing JSON may encode numbers
> differently)
> String: a sequence of zero or more Unicode characters. Strings are delimited
> with double-quotation marks and support a backslash escaping syntax.
>
> So, i think lognorm return wrong JSON. This is must be without double
> quotes:
> { "answer": 42 }
>
> Is this expected behavior?
>
> Also, i did not find how to get IPv6 and IPv4 string
> For example:
>
> IP: 2607:f8b0:4000:807::1006:
> IP: 64.233.160.106
>
> This all of google.com
>
>
> ?????, 10 ???? 2015, 6:15 UTC ?? Chris Schafer <chrisp.schafer at gmail.com>:
>
> Glad I can help. I spent way too much time learning this stuff and I don't
> get to use it often enough :)
> On Tue, Jun 9, 2015 at 11:13 PM David Lang <david at lang.hm> wrote:
>
> On Wed, 10 Jun 2015, ?????? ??????? ??????? wrote:
>
>>
>> Hi!
>> I want to log all from apache. I use custom log in apache that looks like:
>>
>> [2015-06-09 18:27:07 197 NOVT] [192.168.1.67] [192.168.1.67]
>> [192.168.1.254] [818] [/var/www/host/css/button.css] [192.168.1.67]
>> [HTTP/1.1] [1] [-] [GET] [5064] [?v=0] [GET /css/button.css?v=0 HTTP/1.1]
>> [-] [200] [200] [0] [/css/button.css] [hostname.domain] [hostname.domain]
>> [+] [1540] [1138] [" http://hostname.domain/index.html "] ["Mozilla/5.0
>> (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0"]
>>
>> For example, i create test.log that contains:
>> [2015-06-09 16:47:34 830 NOVT]
>>
>> and test.rb:
>> rule=:[%date:date-iso% %time:time-24hr% %microsec:number%
>> %timezone:char-to:]%
>>
>> If i try i get:
>> $ lognormalizer -r test.rb -e json  < test.log
>> { "originalmsg": "[2015-06-09 16:47:34 830 NOVT]", "unparsed-data": "]" }
>>
>> How to parse data between [ and ] ?
>
> you almost have it correct. The only thing that you are missing is that
> char-to
> doesn't 'consume' the matching character, so your rule would need to be:
>
> rule=:[%date:date-iso% %time:time-24hr% %microsec:number%
> %timezone:char-to:]%]
>
> David Lang
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm
>
>
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm
>
>
>
>
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm
>

From fvl at mail.ru  Thu Jun 11 12:36:35 2015
From: fvl at mail.ru (=?UTF-8?B?0KTQsNC00LXQtdCyINCS0LjRgtCw0LvQuNC5INCb0YzQstC+0LLQuNGH?=)
Date: Thu, 11 Jun 2015 13:36:35 +0300
Subject: [Lognorm] =?utf-8?q?Escape_=5B_and_=5D?=
In-Reply-To: <alpine.DEB.2.02.1506101038180.17605@nftneq.ynat.uz>
References: <1433916627.609174282@f94.i.mail.ru>
	<1433919543.213874359@f223.i.mail.ru>
	<alpine.DEB.2.02.1506101038180.17605@nftneq.ynat.uz>
Message-ID: <1434018995.373910225@f391.i.mail.ru>

 Just want to clarify:
i think it would be better to add IPv6 parameter to parse IPv6 address and parameter for both versions of IP.

It can be used, as example, in apache %c - client IP.?It is may contain IPv4 addresss or IPv6 address.


?????, 10 ???? 2015, 10:39 -07:00 ?? David Lang <david at lang.hm>:
>On Wed, 10 Jun 2015, ?????? ??????? ??????? wrote:
>
>> And, probably, there is mismatch:
>>
>> $ cat num.txt 
>> num: 42
>>
>> $ cat num.rb
>> rule=:num: %answer:number%
>>
>> $lognormalizer -r num.rb -e json < num.txt 
>> { "answer": "42" }
>>
>> But if i read on  https://en.wikipedia.org/wiki/JSON
>>
>> JSON's basic types are:
>> *  Number: a signed decimal number that may contain a fractional part and may use exponential  E notation . JSON does not allow non-numbers like  NaN , nor does it make any distinction between integer and floating-point. (Even though JavaScript uses a  double-precision floating-point format for all its numeric values, other languages implementing JSON may encode numbers differently)
>> *  String : a sequence of zero or more  Unicode characters. Strings are delimited with double-quotation marks and support a backslash  escaping syntax.
>>
>> So, i think lognorm return wrong JSON. This is must be without double quotes:
>> { "answer": 42 }
>>
>> Is this expected behavior?
>
>this is the current behavior. The unreleased version in git has a new function 
>'interpret' that can convert strings to numbers among other things.
>
>
>>> Also, i did not find how to get IPv6 and IPv4 string
>>> For example:
>>>
>>> IP: 2607:f8b0:4000:807::1006:
>>> IP: 64.233.160.106
>
>there is an IPv4 type, but not a IPv6 type, it probably should be added.
>
>David Lang
>
>>> This all of google.com
>>>
>>>
>>> ?????, 10 ???? 2015, 6:15 UTC ?? Chris Schafer < chrisp.schafer at gmail.com >:
>>>> Glad I can help. I spent way too much time learning this stuff and I don't get to use it often enough :)
>>>> On Tue, Jun 9, 2015 at 11:13 PM David Lang <  david at lang.hm > wrote:
>>>>> On Wed, 10 Jun 2015, ?????? ??????? ??????? wrote:
>>>>>
>>>>>>
>>>>>> Hi!
>>>>>> I want to log all from apache. I use custom log in apache that looks like:
>>>>>>
>>>>>> [2015-06-09 18:27:07 197 NOVT] [192.168.1.67] [192.168.1.67] [192.168.1.254] [818] [/var/www/host/css/button.css] [192.168.1.67] [HTTP/1.1] [1] [-] [GET] [5064] [?v=0] [GET /css/button.css?v=0 HTTP/1.1] [-] [200] [200] [0] [/css/button.css] [hostname.domain] [hostname.domain] [+] [1540] [1138] ["  http://hostname.domain/index.html "] ["Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0"]
>>>>>>
>>>>>> For example, i create test.log that contains:
>>>>>> [2015-06-09 16:47:34 830 NOVT]
>>>>>>
>>>>>> and test.rb:
>>>>>> rule=:[%date:date-iso% %time:time-24hr% %microsec:number% %timezone:char-to:]%
>>>>>>
>>>>>> If i try i get:
>>>>>> $?lognormalizer -r test.rb -e json? < test.log
>>>>>> { "originalmsg": "[2015-06-09 16:47:34 830 NOVT]", "unparsed-data": "]" }
>>>>>>
>>>>>> How to parse data between [ and ] ?
>>>>>
>>>>> you almost have it correct. The only thing that you are missing is that char-to
>>>>> doesn't 'consume' the matching character, so your rule would need to be:
>>>>>
>>>>> rule=:[%date:date-iso% %time:time-24hr% %microsec:number% %timezone:char-to:]%]
>>>>>
>>>>> David Lang
>>>>> _______________________________________________
>>>>> Lognorm mailing list
>>>>>  Lognorm at lists.adiscon.com
>>>>>  http://lists.adiscon.net/mailman/listinfo/lognorm
>>>>> _______________________________________________
>>>>> Lognorm mailing list
>>>>>  Lognorm at lists.adiscon.com
>>>>>  http://lists.adiscon.net/mailman/listinfo/lognorm
>>>
>>> _______________________________________________
>>> Lognorm mailing list
>>>  Lognorm at lists.adiscon.com
>>>  http://lists.adiscon.net/mailman/listinfo/lognorm
>_______________________________________________
>
>Lognorm mailing list
>
>Lognorm at lists.adiscon.com
>
>http://lists.adiscon.net/mailman/listinfo/lognorm
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20150611/66f0bc1c/attachment-0001.html>