[Lognorm] Any reason we should not support regex-based field-type?

[singh.janmejay]

Hex was just an example. I meant when someone needs to roll out something
very quickly, and it's not supported in the build they have, it's useful to
be able to do it using regex in rulebase but still completely working in
terms of rules, rather than having to resort to a mix of rules and
exec-template calls with regexp based property extractors etc.

It's still the same amount of work, if someone is trying to do something
unsupported by lognorm. So performance effects will still show, just not in
lognorm.

This allows for flexibility and clean way of parsing logs at possibly even
slightly lesser cost then the former approach.

We can support a switch-on flag in conf file(module loading call, may be)
which can be used to enable performance sensitive features.

I personally think warning in logs would be enough though.

--
Regards,
Janmejay

PS: Please blame the typos in this mail on my phone's uncivilized soft
keyboard sporting it's not-so-smart-assist technology.

On Nov 3, 2014 9:22 PM, "Champ Clark III" <cclark at quadrantsec.com> wrote:

>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> I think David means something like:
>
> ./configure --enable-preformance-pigs
>
> I'm not a huge fan of this for a couple of reasons.
>
> In your example,  you want to find hex values.  I'd rather see a lognorm
> "parser" created for this purpose.   I'm afraid if we start with regular
> expressions,  we'll end up mixing rules (RE and non-RE) and it will make
> things very confusing.  I really like the way lognorm's "masking" works.
> I'd rather not see that break.
>
>
>
> On 11/03/2014 07:15 AM, singh.janmejay wrote:
> >
> > We log a warning?
> >
> > --
> > Regards,
> > Janmejay
> >
> > PS: Please blame the typos in this mail on my phone's uncivilized soft
> keyboard sporting it's not-so-smart-assist technology.
> >
> >
> > On Nov 3, 2014 5:37 PM, "David Lang" <david at lang.hm
> <mailto:david at lang.hm> <david at lang.hm>> wrote:
> >
> >     On Mon, 3 Nov 2014, singh.janmejay wrote:
> >
> >         I am thinking of it as a 2nd class field-type.
> >
> >         By that I mean, one gets best performance from 1st class
> supported
> >         field-types, but if for some reason that is not sufficient for
> someone,
> >         they can use a regex-field-type. It may be a little low on
> performance, but
> >         then it unblocks people immediately.
> >
> >         I can do it, just need to know we are not ideologically against
> it.
> >
> >         Kind of construct im thinking of:
> >
> >         %foo:regex:[a-f0-9]+% to match hex-numbers for instance.
> >
> >         Thoughts?
> >
> >
> >     Since this will be such a performance pig compared to the existing
> parse tree, how about requiring a 'enable low performance types' flag or
> something like that to enable it?
> >
> >     There needs to be some good indicator that this is a performance
> problem.
> >
> >     David Lang
> >     _______________________________________________
> >     Lognorm mailing list
> >     Lognorm at lists.adiscon.com <mailto:Lognorm at lists.adiscon.com>
> <Lognorm at lists.adiscon.com>
> >     http://lists.adiscon.net/mailman/listinfo/lognorm
> >
> >     _______________________________________________
> >     Lognorm mailing list
> >     Lognorm at lists.adiscon.com <mailto:Lognorm at lists.adiscon.com>
> <Lognorm at lists.adiscon.com>
> >     http://lists.adiscon.net/mailman/listinfo/lognorm
> >
> >
> >
> > _______________________________________________
> > Lognorm mailing list
> > Lognorm at lists.adiscon.com
> > http://lists.adiscon.net/mailman/listinfo/lognorm
>
>
> - --
> - - Quadrant Information Security
>   Champ Clark III
>   o: 800.538.9357 x 101
>   c: 850.443.2440
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (GNU/Linux)
>
> iQEcBAEBAgAGBQJUV6S0AAoJENnmXt7Lmc3KAwEH/jiwhj/nhFRpvRm7DttrvYTE
> U7kHpjToIMlCSiJqiS4bnvOTS4oTG6mQ5myYzAr16ITTIJQLnJSHmVWBgxlGdUlM
> Kq33I+zYjfUK2Go01PSoLjoE3rgdGa1hptFHVUZREuRkzSP6THtXn8XhexmZzdpH
> 1lC+ZXwNZ9k7FsWm3M027I8zGDKnvZLVdf2UJUElyrNxmWW04ieR3lqJ5qh3Uj8l
> OvTzEDYObghwyS6hThNb1oMz2Sr1AD/mWu+sri1BDKqlPyMkQgYe8KNyI4sKBQHL
> AdCi3TXMvYa8WMb5AOVL4tX0QrOgwNj5mbpeWb2vWrV/S2mkN39TZbE406W91T8=
> =rGsk
> -----END PGP SIGNATURE-----
>
>
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20141103/d3a09063/attachment-0001.html>