[Lognorm] Identifying message types

[Wladimir van der Laan]

Awesome. I think the tag-based approach is very good: it allows for matching
events that are, for example either ssh, login, or fail or a combination of
them. This will be very convenient with a database backend such as MongoDB
which has a built in query for 'give me the records with this and this tag'.

Wladimir

On Wed, Apr 6, 2011 at 3:09 PM, Rainer Gerhards <rgerhards at hq.adiscon.com>wrote:

> It's finally done:
>
> http://blog.gerhards.net/2011/04/log-classification-with-liblognorm.html
>
> This is based on some older CEE ideas and not necessarily inline with what
> comes up. Also, I can think of a couple of more things that would be good
> to
> add. But at least we now have the core functionality.
>
> Feedback, as usual, appreciated. Official release will follow shortly,
> either
> today or tomorrow. Everything already available via git.
>
> Rainer
>
> > -----Original Message-----
> > From: lognorm-bounces at lists.adiscon.com [mailto:lognorm-
> > bounces at lists.adiscon.com] On Behalf Of Wladimir van der Laan
> > Sent: Tuesday, March 22, 2011 12:33 PM
> > To: lognorm
> > Subject: Re: [Lognorm] Identifying message types
> >
> > Hello Rainer,
> >
> > Thanks for the explanation. Looks like I was right in my feeling that
> this
> was
> > missing.
> >
> > I understand your rationale to wait for CEE on this, though. I read their
> spec,
> > and they propose that the identification of a message includes object,
> action
> > and status. But they haven't defined exactly what these should be,
> neither
> > do they give any examples.
> >
> > They still have quite a lot of definition work to to. Hopefully, it won't
> take too
> > long, a standard for logging is very badly needed, and the longer it
> takes,
> the
> > more developers will yet again come up with their own solutions.
> >
> > I'm currently classifying all kinds of events in Zenoss Core, and
> realized
> that
> > when I was defining regexp patterns I could just as well tell it how to
> extract
> > out the interesting information for analysis and more useful
> presentation.
> > Which is how I got to this project.
> >
> > Wladimir
> >
> > BTW: great work on rsyslog.
> >
> >
> > On Tue, Mar 22, 2011 at 10:37 AM, Rainer Gerhards
> > <rgerhards at hq.adiscon.com> wrote:
> >
> >
> >       Hi Wladimir,
> >
> >       This is a good question and you are abosultely right -- this is
> currently
> >       missing. In fact, the speace in front of the colon inside the
> rulebase is
> >       reserved for tags, which is the classification you are looking for.
> >       Liblognorm is in its infancy, though already quite useful in its
> current
> >       state. I have paused development a bit for two reasons:
> >
> >       a) CEE needs to sort out some things -- I'd prefer to have some
> issues
> > solved
> >       before continuing (and re-doing some work).
> >       b) devel prio -- right now I am working hard on getting a new
> stable
> > v5
> >       rsyslog out, and this is taking quite some toll
> >
> >       The feature you are asking for is definitely on the today list, and
> I
> > hope to
> >       be able to work more on liblognorm within the next couple of weeks
> > (this year
> >       has been very busy - and will be - at least until mid-april).
> >
> >       Rainer
> >
> >
> >       > -----Original Message-----
> >       > From: lognorm-bounces at lists.adiscon.com [mailto:lognorm-
> >       > bounces at lists.adiscon.com] On Behalf Of Wladimir van der Laan
> >       > Sent: Monday, March 21, 2011 7:00 PM
> >       > To: lognorm at lists.adiscon.com
> >       > Subject: [Lognorm] Identifying message types
> >       >
> >       > Hello,
> >       >
> >       > I have a question about the usage of lognorm. As I understand,
> the
> >       > program extracts data fields from log messages in text format, by
> > means
> >       > of examples from a ruleset file. The output is represented as
> > metadata
> >       > key/value pairs.
> >       >
> >       > But as far as I can see, it outputs no identifier as to what kind
> of
> >       > message the log line represents. For automated log processing,
> one
> >       > would also need to identify the message, for example, as failed
> >       > authentication, or dhcp request, etc.
> >       >
> >       > Am I overlooking something? Is it possible to add a message type
> > field
> >       > in a ruleset?
> >       >
> >       > Greetings,
> >       > Wladimir
> >       >
> >
> >
> >       _______________________________________________
> >       Lognorm mailing list
> >       Lognorm at lists.adiscon.com
> >       http://lists.adiscon.net/mailman/listinfo/lognorm
> >
> >
>
> _______________________________________________
> Lognorm mailing list
> Lognorm at lists.adiscon.com
> http://lists.adiscon.net/mailman/listinfo/lognorm
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.adiscon.net/pipermail/lognorm/attachments/20110406/4c9a9a13/attachment-0001.htm>