[Lognorm] Special characters revisited
Champ Clark III (Quadrant)
cclark at quadrantsec.com
Wed Jul 4 21:50:55 CEST 2012
Rainer,
Thanks for Looking into this. I've had similar issues as well. Look forward to reading more about this.
Champ Clark III - Quadrant information security
----- Reply message -----
From: "James Lay" <jlay at slave-tothe-box.net>
Date: Wed, Jul 4, 2012 12:44
Subject: [Lognorm] Special characters revisited
To: "lognorm" <lognorm at lists.adiscon.com>
On Jul 4, 2012, at 9:32 AM, Rainer Gerhards wrote:
>
>
>> -----Original Message-----
>> From: lognorm-bounces at lists.adiscon.com [mailto:lognorm-
>> bounces at lists.adiscon.com] On Behalf Of James Lay
>> Sent: Wednesday, July 04, 2012 4:10 PM
>> To: lognorm
>> Subject: Re: [Lognorm] Special characters revisited
>>
>> On Jun 13, 2012, at 7:11 AM, James Lay wrote:
>>> On May 25, 2012, at 12:44 PM, James Lay wrote:
>>>> On Fri, 25 May 2012 12:33:15 -0600, James Lay wrote:
>>>>> Hey all!
>>>>>
>>>>> So...here's where I'm at. Here's my rulebase:
>>>>>
>>>>> prefix=
>>>>> rule=: %-:word% request discarded from
>>>>> %src-ip:ipv4%/%src-port:number% to
>>>>> %-:word%:%dst-ip:ipv4%/%dst-port:number%
>>>>>
>>>>>
>>>>> Sample firewall hit (quotes added for clarity):
>>>>>
>>>>> " TCP request discarded from 96.18.101.250/1077 to Interface:my_ip/443"
>>>>>
>>>>> and my result:
>>>>>
>>>>> [cee at 115 originalmsg=" TCP request discarded from 96.18.101.250/1077
>>>>> to Interface:my_ip/443" unparsed-data=""]
>>>>>
>>>>> I've tried the below (thinking the : would be included in word)
>>>>> rule=: %-:word% request discarded from
>>>>> %src-ip:ipv4%/%src-port:number% to
>>>>> %-:word%%dst-ip:ipv4%/%dst-port:number%
>>>>>
>>>>> And I get the same result.
>>>>>
>>>>> I've also tried:
>>>>> rule=: %-:word% request discarded from
>>>>> %src-ip:ipv4%/%src-port:number% to
>>>>> Interface:%dst-ip:ipv4%/%dst-port:number%
>>>>>
>>>>> And this works, so I'm pretty sure it's the : that is my issue. A
>>>>> full -vv below. Any thoughts on how to get this to go? Thanks all:
>>>>>
>>>>> James
>>>>>
>>>>> liblognorm: read sample line: 'prefix='
>>>>> liblognorm: read sample line: 'rule=: %-:word% request discarded from
>>>>> %src-ip:ipv4%/%src-port:number% to
>>>>> %-:word%:%dst-ip:ipv4%/%dst-port:number%'
>>>>> liblognorm: sample line to add: ': %-:word% request discarded from
>>>>> %src-ip:ipv4%/%src-port:number% to
>>>>> %-:word%:%dst-ip:ipv4%/%dst-port:number%'
>>>>>
>>>>> liblognorm: addSampToTree 0 of 108
>>>>> liblognorm: parsed literal: ' '
>>>>> liblognorm: buildPTree: begin at 0x9308030, offs 0
>>>>> liblognorm: case 3.1
>>>>> liblognorm: addPTree: offs 0
>>>>> liblognorm: setPrefix lenBuf 1, offs 0
>>>>> liblognorm: addSampToTree 1 of 108
>>>>> liblognorm: parsed field: '-'
>>>>> liblognorm: got new subtree 0x9308810
>>>>> liblognorm: prev subtree 0x9308030
>>>>> liblognorm: new subtree 0x9308810
>>>>> liblognorm: addSampToTree 9 of 108
>>>>> liblognorm: parsed literal: ' request discarded from '
>>>>> liblognorm: buildPTree: begin at 0x9308810, offs 0
>>>>> liblognorm: case 3.1
>>>>> liblognorm: addPTree: offs 0
>>>>> liblognorm: setPrefix lenBuf 24, offs 0
>>>>> liblognorm: addSampToTree 33 of 108
>>>>> liblognorm: parsed field: 'src-ip'
>>>>> liblognorm: got new subtree 0x9308ca0
>>>>> liblognorm: prev subtree 0x9308810
>>>>> liblognorm: new subtree 0x9308ca0
>>>>> liblognorm: addSampToTree 46 of 108
>>>>> liblognorm: parsed literal: '/'
>>>>> liblognorm: buildPTree: begin at 0x9308ca0, offs 0
>>>>> liblognorm: case 3.1
>>>>> liblognorm: addPTree: offs 0
>>>>> liblognorm: setPrefix lenBuf 1, offs 0
>>>>> liblognorm: addSampToTree 47 of 108
>>>>> liblognorm: parsed field: 'src-port'
>>>>> liblognorm: got new subtree 0x9309110
>>>>> liblognorm: prev subtree 0x9308ca0
>>>>> liblognorm: new subtree 0x9309110
>>>>> liblognorm: addSampToTree 64 of 108
>>>>> liblognorm: parsed literal: ' to '
>>>>> liblognorm: buildPTree: begin at 0x9309110, offs 0
>>>>> liblognorm: case 3.1
>>>>> liblognorm: addPTree: offs 0
>>>>> liblognorm: setPrefix lenBuf 4, offs 0
>>>>> liblognorm: addSampToTree 68 of 108
>>>>> liblognorm: parsed field: '-'
>>>>> liblognorm: got new subtree 0x9309580
>>>>> liblognorm: prev subtree 0x9309110
>>>>> liblognorm: new subtree 0x9309580
>>>>> liblognorm: addSampToTree 76 of 108
>>>>> liblognorm: parsed literal: ':'
>>>>> liblognorm: buildPTree: begin at 0x9309580, offs 0
>>>>> liblognorm: case 3.1
>>>>> liblognorm: addPTree: offs 0
>>>>> liblognorm: setPrefix lenBuf 1, offs 0
>>>>> liblognorm: addSampToTree 77 of 108
>>>>> liblognorm: parsed field: 'dst-ip'
>>>>> liblognorm: got new subtree 0x93099f0
>>>>> liblognorm: prev subtree 0x9309580
>>>>> liblognorm: new subtree 0x93099f0
>>>>> liblognorm: addSampToTree 90 of 108
>>>>> liblognorm: parsed literal: '/'
>>>>> liblognorm: buildPTree: begin at 0x93099f0, offs 0
>>>>> liblognorm: case 3.1
>>>>> liblognorm: addPTree: offs 0
>>>>> liblognorm: setPrefix lenBuf 1, offs 0
>>>>> liblognorm: addSampToTree 91 of 108
>>>>> liblognorm: parsed field: 'dst-port'
>>>>> liblognorm: got new subtree 0x9309e60
>>>>> liblognorm: prev subtree 0x93099f0
>>>>> liblognorm: new subtree 0x9309e60
>>>>> liblognorm: end addSampToTree 108 of 108
>>>>> number of tree nodes: 7
>>>>> To normalize: ' TCP request discarded from 96.18.101.250/1077 to
>>>>> Interface:my_ip/443'
>>>>> liblognorm: 0: prefix compare ' ', ' '
>>>>> liblognorm: 1: prefix compare succeeded, still valid
>>>>> liblognorm: 1:trying parser for field '-': 0xb786c450
>>>>> liblognorm: potential hit, trying subtree
>>>>> liblognorm: 4: prefix compare ' ', ' '
>>>>> liblognorm: 5: prefix compare 'r', 'r'
>>>>> liblognorm: 6: prefix compare 'e', 'e'
>>>>> liblognorm: 7: prefix compare 'q', 'q'
>>>>> liblognorm: 8: prefix compare 'u', 'u'
>>>>> liblognorm: 9: prefix compare 'e', 'e'
>>>>> liblognorm: 10: prefix compare 's', 's'
>>>>> liblognorm: 11: prefix compare 't', 't'
>>>>> liblognorm: 12: prefix compare ' ', ' '
>>>>> liblognorm: 13: prefix compare 'd', 'd'
>>>>> liblognorm: 14: prefix compare 'i', 'i'
>>>>> liblognorm: 15: prefix compare 's', 's'
>>>>> liblognorm: 16: prefix compare 'c', 'c'
>>>>> liblognorm: 17: prefix compare 'a', 'a'
>>>>> liblognorm: 18: prefix compare 'r', 'r'
>>>>> liblognorm: 19: prefix compare 'd', 'd'
>>>>> liblognorm: 20: prefix compare 'e', 'e'
>>>>> liblognorm: 21: prefix compare 'd', 'd'
>>>>> liblognorm: 22: prefix compare ' ', ' '
>>>>> liblognorm: 23: prefix compare 'f', 'f'
>>>>> liblognorm: 24: prefix compare 'r', 'r'
>>>>> liblognorm: 25: prefix compare 'o', 'o'
>>>>> liblognorm: 26: prefix compare 'm', 'm'
>>>>> liblognorm: 27: prefix compare ' ', ' '
>>>>> liblognorm: 28: prefix compare succeeded, still valid
>>>>> liblognorm: 28:trying parser for field 'src-ip': 0xb786bdb0
>>>>> liblognorm: potential hit, trying subtree
>>>>> liblognorm: 41: prefix compare '/', '/'
>>>>> liblognorm: 42: prefix compare succeeded, still valid
>>>>> liblognorm: 42:trying parser for field 'src-port': 0xb786c510
>>>>> liblognorm: potential hit, trying subtree
>>>>> liblognorm: 46: prefix compare ' ', ' '
>>>>> liblognorm: 47: prefix compare 't', 't'
>>>>> liblognorm: 48: prefix compare 'o', 'o'
>>>>> liblognorm: 49: prefix compare ' ', ' '
>>>>> liblognorm: 50: prefix compare succeeded, still valid
>>>>> liblognorm: 50:trying parser for field '-': 0xb786c450
>>>>> liblognorm: potential hit, trying subtree
>>>>> liblognorm: 74 returns -1
>>>>> liblognorm: 50 nonmatch, backtracking required, left=-1
>>>>> liblognorm: 50 no field, trying subtree char 'I': (nil)
>>>>> liblognorm: 50 returns -1
>>>>> liblognorm: 42 nonmatch, backtracking required, left=-1
>>>>> liblognorm: 42 no field, trying subtree char '1': (nil)
>>>>> liblognorm: 42 returns -1
>>>>> liblognorm: 28 nonmatch, backtracking required, left=-1
>>>>> liblognorm: 28 no field, trying subtree char '9': (nil)
>>>>> liblognorm: 28 returns -1
>>>>> liblognorm: 1 nonmatch, backtracking required, left=-1
>>>>> liblognorm: 1 no field, trying subtree char 'T': (nil)
>>>>> liblognorm: 1 returns -1
>>>>> liblognorm: final result for normalizer: left -1, endNode 0x9309580
>>>>> normalized: '[cee at 115 originalmsg=" TCP request discarded from
>>>>> 96.18.101.250/1077 to Interface:my_ip/443" unparsed-data=""]'
>>>>> [cee at 115 originalmsg=" TCP request discarded from 96.18.101.250/1077
>>>>> to Interface:my_ip/443" unparsed-data=""]
>>>>>
>>>>> _______________________________________________
>>>>> Lognorm mailing list
>>>>> Lognorm at lists.adiscon.com
>>>>> http://lists.adiscon.net/mailman/listinfo/lognorm
>>>>
>>>> An interesting tidbit...with the below:
>>>>
>>>>
>>>> " TCP request discarded from 96.18.101.250/1077 to 1234:my_ip/443"
>>>>
>>>> and
>>>>
>>>> rule=: %-:word% request discarded from %src-ip:ipv4%/%src-port:number%
>> to %-:number%:%dst-ip:ipv4%/%dst-port:number%
>>>>
>>>> This works and correctly processes. Hope that helps.
>>>>
>>>> James
>>>>
>>>> _______________________________________________
>>>> Lognorm mailing list
>>>> Lognorm at lists.adiscon.com
>>>> http://lists.adiscon.net/mailman/listinfo/lognorm
>>>
>>> Any movement on this? I'm at an impasse until this is addressed...thank you.
>>>
>>> James
>>> _______________________________________________
>>> Lognorm mailing list
>>> Lognorm at lists.adiscon.com
>>> http://lists.adiscon.net/mailman/listinfo/lognorm
>>
>> Would have been nice to at least get some response of some kind on this...
>
> Oh, damn, sorry - I have totally lost track of this. I need to look at another issue today, but will try to tweak it in between the other stuff ASAP.
>
> Rainer
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm
Thanks Rainer…I'll give this a shot when I get to work tomorrow.
James
_______________________________________________
Lognorm mailing list
Lognorm at lists.adiscon.com
http://lists.adiscon.net/mailman/listinfo/lognorm
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20120704/100fabab/attachment-0001.htm>
More information about the Lognorm
mailing list