[Lognorm] Question on special characters

Fri Dec 2 16:45:47 CET 2011

Ah for.....yea check this out:

liblognorm: read sample line: 'prefix='
liblognorm: read sample line: 'rule=: Test %-:word% '
liblognorm: sample line to add: ': Test %-:word% '

liblognorm: addSampToTree 0 of 15
liblognorm: parsed literal: ' Test '
liblognorm: buildPTree: begin at 0x8bcd030, offs 0
liblognorm: case 3.1
liblognorm: addPTree: offs 0
liblognorm: setPrefix lenBuf 6, offs 0
liblognorm: addSampToTree 6 of 15
liblognorm: parsed field: '-'
liblognorm: got new subtree 0x8bcd7a0
liblognorm: prev subtree 0x8bcd030
liblognorm: new subtree 0x8bcd7a0
liblognorm: addSampToTree 14 of 15
liblognorm: parsed literal: ' '
liblognorm: buildPTree: begin at 0x8bcd7a0, offs 0
liblognorm: case 3.1
liblognorm: addPTree: offs 0
liblognorm: setPrefix lenBuf 1, offs 0
liblognorm: end addSampToTree 15 of 15
number of tree nodes: 2
To normalize: ' Test one) '
liblognorm: 0: prefix compare ' ', ' '
liblognorm: 1: prefix compare 'T', 'T'
liblognorm: 2: prefix compare 'e', 'e'
liblognorm: 3: prefix compare 's', 's'
liblognorm: 4: prefix compare 't', 't'
liblognorm: 5: prefix compare ' ', ' '
liblognorm: 6: prefix compare succeeded, still valid
liblognorm: 6:trying parser for field '-': 0xb7795930
liblognorm: potential hit, trying subtree
liblognorm: 10: prefix compare ' ', ' '
liblognorm: 11: prefix compare succeeded, still valid
liblognorm: 11 returns 0
liblognorm: 6: parser matches at 10
liblognorm: 6 returns 0
liblognorm: final result for normalizer: left 0, endNode 0x8bcd7a0,
isTerminal 1, tagbucket (nil)
normalized: '[cee at 115 -="one)"]'
[cee at 115 -="one)"]

That's what it was..."one)" is viewed as one word!  Nice...thanks for
the assist Rainer.  Hope you get a break at some point in time.

James

> -----Original Message-----
> From: lognorm-bounces at lists.adiscon.com
[mailto:lognorm-bounces at lists.adiscon.com] On Behalf Of Lay, James
> Sent: Friday, December 02, 2011 8:43 AM
> To: lognorm
> Subject: Re: [Lognorm] Question on special characters
> 
> Thanks Rainer...I'll look at the char-upto option and see what happens
> :)
> 
> > -----Original Message-----
> > From: lognorm-bounces at lists.adiscon.com
> [mailto:lognorm-bounces at lists.adiscon.com] On Behalf Of Rainer
Gerhards
> > Sent: Friday, December 02, 2011 8:42 AM
> > To: lognorm
> > Subject: Re: [Lognorm] Question on special characters
> >
> > > -----Original Message-----
> > > From: lognorm-bounces at lists.adiscon.com [mailto:lognorm-
> > > bounces at lists.adiscon.com] On Behalf Of Lay, James
> > > Sent: Friday, December 02, 2011 4:38 PM
> > > To: lognorm
> > > Subject: Re: [Lognorm] Question on special characters
> > >
> > > Thanks Rainer....tried with -vv and even -vvv...got the same info
as
> below.
> > I'll
> > > move on to other devices and backtrack to this one later.
> >
> > Maybe my fault and you need to add some options ;) Anyhow, I had a
> quick
> > look. I think (not 100% sure) %word% is defined a whitespace
delimited
> > sequence. If so, "one)" is one word! There is a char-upto syntax
where
> you
> > can say "evertything up to ")". This may help here. All from the
back
> of my
> > head and unverified...
> >
> > rainer
> > >
> > > James
> > >
> > > > -----Original Message-----
> > > > From: lognorm-bounces at lists.adiscon.com
> > > [mailto:lognorm-bounces at lists.adiscon.com] On Behalf Of Rainer
> Gerhards
> > > > Sent: Friday, December 02, 2011 8:34 AM
> > > > To: lognorm
> > > > Subject: Re: [Lognorm] Question on special characters
> > > >
> > > > I'll have a look soon, just wanted to add that -vv gives you
even
> more
> > > info
> > > > (probably more than you ever want ;))
> > > > rainer
> > > >
> > > > > -----Original Message-----
> > > > > From: lognorm-bounces at lists.adiscon.com [mailto:lognorm-
> > > > > bounces at lists.adiscon.com] On Behalf Of Lay, James
> > > > > Sent: Friday, December 02, 2011 4:30 PM
> > > > > To: lognorm
> > > > > Subject: Re: [Lognorm] Question on special characters
> > > > >
> > > > > LoL...36 seems a little beefy ;)  Ok...here's the -v
> output....very
> > > > interesting!
> > > > > Thanks again Rainer.
> > > > >
> > > > > liblognorm: read sample line: 'prefix='
> > > > > liblognorm: read sample line: 'rule=: Test %-:word%) '
> > > > > liblognorm: sample line to add: ': Test %-:word%) '
> > > > >
> > > > > liblognorm: addSampToTree 0 of 16
> > > > > liblognorm: parsed literal: ' Test '
> > > > > liblognorm: buildPTree: begin at 0x8c15030, offs 0
> > > > > liblognorm: case 3.1
> > > > > liblognorm: addPTree: offs 0
> > > > > liblognorm: setPrefix lenBuf 6, offs 0
> > > > > liblognorm: addSampToTree 6 of 16
> > > > > liblognorm: parsed field: '-'
> > > > > liblognorm: got new subtree 0x8c157a0
> > > > > liblognorm: prev subtree 0x8c15030
> > > > > liblognorm: new subtree 0x8c157a0
> > > > > liblognorm: addSampToTree 14 of 16
> > > > > liblognorm: parsed literal: ') '
> > > > > liblognorm: buildPTree: begin at 0x8c157a0, offs 0
> > > > > liblognorm: case 3.1
> > > > > liblognorm: addPTree: offs 0
> > > > > liblognorm: setPrefix lenBuf 2, offs 0
> > > > > liblognorm: end addSampToTree 16 of 16 number of tree nodes: 2
> To
> > > > > normalize: ' Test one) '
> > > > > liblognorm: 0: prefix compare ' ', ' '
> > > > > liblognorm: 1: prefix compare 'T', 'T'
> > > > > liblognorm: 2: prefix compare 'e', 'e'
> > > > > liblognorm: 3: prefix compare 's', 's'
> > > > > liblognorm: 4: prefix compare 't', 't'
> > > > > liblognorm: 5: prefix compare ' ', ' '
> > > > > liblognorm: 6: prefix compare succeeded, still valid
> > > > > liblognorm: 6:trying parser for field '-': 0xb77c4930
> > > > > liblognorm: potential hit, trying subtree
> > > > > liblognorm: 10: prefix compare ' ', ')'
> > > > > liblognorm: 10 returns 1
> > > > > liblognorm: 6 nonmatch, backtracking required, left=1
> > > > > liblognorm: 6 no field, trying subtree char 'o': (nil)
> > > > > liblognorm: 6 returns 1
> > > > > liblognorm: final result for normalizer: left 1, endNode
> 0xbfc09dd0
> > > > > normalized: '[cee at 115 originalmsg=" Test one) "
unparsed-data="
> "]'
> > > > > [cee at 115 originalmsg=" Test one) " unparsed-data=" "]
> > > > >
> > > > > James
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: lognorm-bounces at lists.adiscon.com
> > > > > [mailto:lognorm-bounces at lists.adiscon.com] On Behalf Of Rainer
> > > Gerhards
> > > > > > Sent: Friday, December 02, 2011 8:19 AM
> > > > > > To: lognorm
> > > > > > Subject: Re: [Lognorm] Question on special characters
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: lognorm-bounces at lists.adiscon.com [mailto:lognorm-
> > > > > > > bounces at lists.adiscon.com] On Behalf Of Lay, James
> > > > > > > Sent: Friday, December 02, 2011 4:17 PM
> > > > > > > To: lognorm
> > > > > > > Subject: Re: [Lognorm] Question on special characters
> > > > > > >
> > > > > > > Thanks Rainer....is there anything I can do on my end to
> > > > > troubleshoot or
> > > > > > > make things easier?
> > > > > >
> > > > > > Get me a 36 hr day ;-) Let me check on the original
question,
> > > maybe I
> > > > > see it.
> > > > > > Did you try with liblognorm's normalizer tool in verbose
(-v)
> mode
> > > -
> > > > > that is
> > > > > > often quite educating.
> > > > > >
> > > > > > Rainer
> > > > > >
> > > > > > >
> > > > > > > James
> > > > > > >
> > > > > > > > -----Original Message-----
> > > > > > > > From: lognorm-bounces at lists.adiscon.com
> > > > > > > [mailto:lognorm-bounces at lists.adiscon.com] On Behalf Of
> Rainer
> > > > > Gerhards
> > > > > > > > Sent: Friday, December 02, 2011 7:54 AM
> > > > > > > > To: lognorm
> > > > > > > > Subject: Re: [Lognorm] Question on special characters
> > > > > > > >
> > > > > > > > Sorry...all of my time has been taken by that journald
> > > proposal. I
> > > > > > > hope to
> > > > > > > > resume regular work next week...
> > > > > > > >
> > > > > > > > rainer
> > > > > > > >
> > > > > > > > > -----Original Message-----
> > > > > > > > > From: lognorm-bounces at lists.adiscon.com
[mailto:lognorm-
> > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Lay, James
> > > > > > > > > Sent: Friday, December 02, 2011 3:52 PM
> > > > > > > > > To: lognorm at lists.adiscon.com
> > > > > > > > > Subject: Re: [Lognorm] Question on special characters
> > > > > > > > >
> > > > > > > > > Any movement on this?  I am unable to move forward
with
> rule
> > > > > > > creation for
> > > > > > > > > one of my devices until this is ironed out.  Thank
you.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > James
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > From: lognorm-bounces at lists.adiscon.com
[mailto:lognorm-
> > > > > > > > > bounces at lists.adiscon.com] On Behalf Of Lay, James
> > > > > > > > > Sent: Thursday, November 24, 2011 11:32 AM
> > > > > > > > > To: lognorm at lists.adiscon.com
> > > > > > > > > Subject: [Lognorm] Question on special characters
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Hey all!
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > So...I think I'm getting down to the bottom of
something
> > > that
> > > > > I've
> > > > > > > had an
> > > > > > > > > issue with.  Here's some tests:
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Log contents to pass to normalizer, blick.txt:
> > > > > > > > >
> > > > > > > > > Test one)
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Rulebase file blick-rulebase:
> > > > > > > > >
> > > > > > > > > prefix=
> > > > > > > > >
> > > > > > > > > rule=: Test one)
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > normalizer -r blick-rulebase < blick.txt
> > > > > > > > >
> > > > > > > > > this works, and returns nothing, since no normalizing
> was
> > > > > required
> > > > > > > (as I
> > > > > > > > > understand it).
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Now...if I make the below change to the blick-rulebase
> file:
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > prefix=
> > > > > > > > >
> > > > > > > > > rule=: Test %-:word%)
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > normalizer -r blick-rulebase < blick.txt
> > > > > > > > >
> > > > > > > > > [cee at 115 originalmsg=" Test one)" unparsed-data=" "]
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Then it looks like something isn't working.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > If I remove the ")" in both blick.txt and
blick-rulebase
> to
> > > > > reflect:
> > > > > > > > >
> > > > > > > > > Test one
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > prefix=
> > > > > > > > >
> > > > > > > > > rule=: Test %-:word%
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > then it works:
> > > > > > > > >
> > > > > > > > > normalizer -r blick-rulebase < blick.txt
> > > > > > > > >
> > > > > > > > > [cee at 115 -="one"]
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > This seems to happen with matching %word% within
> > > parenthesis.
> > > > > Is
> > > > > > > there
> > > > > > > > > something I can do to check this on my end?  Thank
you.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > James
> > > > > > > >
> > > > > > > > _______________________________________________
> > > > > > > > Lognorm mailing list
> > > > > > > > Lognorm at lists.adiscon.com
> > > > > > > > http://lists.adiscon.net/mailman/listinfo/lognorm
> > > > > > > _______________________________________________
> > > > > > > Lognorm mailing list
> > > > > > > Lognorm at lists.adiscon.com
> > > > > > > http://lists.adiscon.net/mailman/listinfo/lognorm
> > > > > > _______________________________________________
> > > > > > Lognorm mailing list
> > > > > > Lognorm at lists.adiscon.com
> > > > > > http://lists.adiscon.net/mailman/listinfo/lognorm
> > > > > _______________________________________________
> > > > > Lognorm mailing list
> > > > > Lognorm at lists.adiscon.com
> > > > > http://lists.adiscon.net/mailman/listinfo/lognorm
> > > > _______________________________________________
> > > > Lognorm mailing list
> > > > Lognorm at lists.adiscon.com
> > > > http://lists.adiscon.net/mailman/listinfo/lognorm
> > > _______________________________________________
> > > Lognorm mailing list
> > > Lognorm at lists.adiscon.com
> > > http://lists.adiscon.net/mailman/listinfo/lognorm
> > _______________________________________________
> > Lognorm mailing list
> > Lognorm at lists.adiscon.com
> > http://lists.adiscon.net/mailman/listinfo/lognorm
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm