[Lognorm] Special characters revisited

[James Lay]

Hey all!

So...here's where I'm at.  Here's my rulebase:

prefix=
rule=: %-:word% request discarded from %src-ip:ipv4%/%src-port:number% 
to %-:word%:%dst-ip:ipv4%/%dst-port:number%


Sample firewall hit (quotes added for clarity):

" TCP request discarded from 96.18.101.250/1077 to Interface:my_ip/443"

and my result:

[cee at 115 originalmsg=" TCP request discarded from 96.18.101.250/1077 to 
Interface:my_ip/443" unparsed-data=""]

I've tried the below (thinking the : would be included in word)
rule=: %-:word% request discarded from %src-ip:ipv4%/%src-port:number% 
to %-:word%%dst-ip:ipv4%/%dst-port:number%

And I get the same result.

I've also tried:
rule=: %-:word% request discarded from %src-ip:ipv4%/%src-port:number% 
to Interface:%dst-ip:ipv4%/%dst-port:number%

And this works, so I'm pretty sure it's the : that is my issue.  A full 
-vv below.  Any thoughts on how to get this to go?  Thanks all:

James

liblognorm: read sample line: 'prefix='
liblognorm: read sample line: 'rule=: %-:word% request discarded from 
%src-ip:ipv4%/%src-port:number% to 
%-:word%:%dst-ip:ipv4%/%dst-port:number%'
liblognorm: sample line to add: ': %-:word% request discarded from 
%src-ip:ipv4%/%src-port:number% to 
%-:word%:%dst-ip:ipv4%/%dst-port:number%'

liblognorm: addSampToTree 0 of 108
liblognorm: parsed literal: ' '
liblognorm: buildPTree: begin at 0x9308030, offs 0
liblognorm: case 3.1
liblognorm: addPTree: offs 0
liblognorm: setPrefix lenBuf 1, offs 0
liblognorm: addSampToTree 1 of 108
liblognorm: parsed field: '-'
liblognorm: got new subtree 0x9308810
liblognorm: prev subtree 0x9308030
liblognorm: new subtree 0x9308810
liblognorm: addSampToTree 9 of 108
liblognorm: parsed literal: ' request discarded from '
liblognorm: buildPTree: begin at 0x9308810, offs 0
liblognorm: case 3.1
liblognorm: addPTree: offs 0
liblognorm: setPrefix lenBuf 24, offs 0
liblognorm: addSampToTree 33 of 108
liblognorm: parsed field: 'src-ip'
liblognorm: got new subtree 0x9308ca0
liblognorm: prev subtree 0x9308810
liblognorm: new subtree 0x9308ca0
liblognorm: addSampToTree 46 of 108
liblognorm: parsed literal: '/'
liblognorm: buildPTree: begin at 0x9308ca0, offs 0
liblognorm: case 3.1
liblognorm: addPTree: offs 0
liblognorm: setPrefix lenBuf 1, offs 0
liblognorm: addSampToTree 47 of 108
liblognorm: parsed field: 'src-port'
liblognorm: got new subtree 0x9309110
liblognorm: prev subtree 0x9308ca0
liblognorm: new subtree 0x9309110
liblognorm: addSampToTree 64 of 108
liblognorm: parsed literal: ' to '
liblognorm: buildPTree: begin at 0x9309110, offs 0
liblognorm: case 3.1
liblognorm: addPTree: offs 0
liblognorm: setPrefix lenBuf 4, offs 0
liblognorm: addSampToTree 68 of 108
liblognorm: parsed field: '-'
liblognorm: got new subtree 0x9309580
liblognorm: prev subtree 0x9309110
liblognorm: new subtree 0x9309580
liblognorm: addSampToTree 76 of 108
liblognorm: parsed literal: ':'
liblognorm: buildPTree: begin at 0x9309580, offs 0
liblognorm: case 3.1
liblognorm: addPTree: offs 0
liblognorm: setPrefix lenBuf 1, offs 0
liblognorm: addSampToTree 77 of 108
liblognorm: parsed field: 'dst-ip'
liblognorm: got new subtree 0x93099f0
liblognorm: prev subtree 0x9309580
liblognorm: new subtree 0x93099f0
liblognorm: addSampToTree 90 of 108
liblognorm: parsed literal: '/'
liblognorm: buildPTree: begin at 0x93099f0, offs 0
liblognorm: case 3.1
liblognorm: addPTree: offs 0
liblognorm: setPrefix lenBuf 1, offs 0
liblognorm: addSampToTree 91 of 108
liblognorm: parsed field: 'dst-port'
liblognorm: got new subtree 0x9309e60
liblognorm: prev subtree 0x93099f0
liblognorm: new subtree 0x9309e60
liblognorm: end addSampToTree 108 of 108
number of tree nodes: 7
To normalize: ' TCP request discarded from 96.18.101.250/1077 to 
Interface:my_ip/443'
liblognorm: 0: prefix compare ' ', ' '
liblognorm: 1: prefix compare succeeded, still valid
liblognorm: 1:trying parser for field '-': 0xb786c450
liblognorm: potential hit, trying subtree
liblognorm: 4: prefix compare ' ', ' '
liblognorm: 5: prefix compare 'r', 'r'
liblognorm: 6: prefix compare 'e', 'e'
liblognorm: 7: prefix compare 'q', 'q'
liblognorm: 8: prefix compare 'u', 'u'
liblognorm: 9: prefix compare 'e', 'e'
liblognorm: 10: prefix compare 's', 's'
liblognorm: 11: prefix compare 't', 't'
liblognorm: 12: prefix compare ' ', ' '
liblognorm: 13: prefix compare 'd', 'd'
liblognorm: 14: prefix compare 'i', 'i'
liblognorm: 15: prefix compare 's', 's'
liblognorm: 16: prefix compare 'c', 'c'
liblognorm: 17: prefix compare 'a', 'a'
liblognorm: 18: prefix compare 'r', 'r'
liblognorm: 19: prefix compare 'd', 'd'
liblognorm: 20: prefix compare 'e', 'e'
liblognorm: 21: prefix compare 'd', 'd'
liblognorm: 22: prefix compare ' ', ' '
liblognorm: 23: prefix compare 'f', 'f'
liblognorm: 24: prefix compare 'r', 'r'
liblognorm: 25: prefix compare 'o', 'o'
liblognorm: 26: prefix compare 'm', 'm'
liblognorm: 27: prefix compare ' ', ' '
liblognorm: 28: prefix compare succeeded, still valid
liblognorm: 28:trying parser for field 'src-ip': 0xb786bdb0
liblognorm: potential hit, trying subtree
liblognorm: 41: prefix compare '/', '/'
liblognorm: 42: prefix compare succeeded, still valid
liblognorm: 42:trying parser for field 'src-port': 0xb786c510
liblognorm: potential hit, trying subtree
liblognorm: 46: prefix compare ' ', ' '
liblognorm: 47: prefix compare 't', 't'
liblognorm: 48: prefix compare 'o', 'o'
liblognorm: 49: prefix compare ' ', ' '
liblognorm: 50: prefix compare succeeded, still valid
liblognorm: 50:trying parser for field '-': 0xb786c450
liblognorm: potential hit, trying subtree
liblognorm: 74 returns -1
liblognorm: 50 nonmatch, backtracking required, left=-1
liblognorm: 50 no field, trying subtree char 'I': (nil)
liblognorm: 50 returns -1
liblognorm: 42 nonmatch, backtracking required, left=-1
liblognorm: 42 no field, trying subtree char '1': (nil)
liblognorm: 42 returns -1
liblognorm: 28 nonmatch, backtracking required, left=-1
liblognorm: 28 no field, trying subtree char '9': (nil)
liblognorm: 28 returns -1
liblognorm: 1 nonmatch, backtracking required, left=-1
liblognorm: 1 no field, trying subtree char 'T': (nil)
liblognorm: 1 returns -1
liblognorm: final result for normalizer: left -1, endNode 0x9309580
normalized: '[cee at 115 originalmsg=" TCP request discarded from 
96.18.101.250/1077 to Interface:my_ip/443" unparsed-data=""]'
[cee at 115 originalmsg=" TCP request discarded from 96.18.101.250/1077 to 
Interface:my_ip/443" unparsed-data=""]